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EXECUTIVE SUMMARY 


In 2014, The Wallace Foundation 
launched the Principal Supervisor Initiative 
(PSD, a four-year, $24 million-dollar effort to 
redefine principal supervision in six urban 
school districts. The initiative sought to help 
districts transform a position traditionally 
focused on administration, operations, and 
compliance to one dedicated to developing and 
supporting principals to improve instruction in Minneapolis Publi Schools: Minnesota 
schools. 


Six urban school districts participated in the 
Principal Supervisor Initiative: 


Broward County Public Schools, Florida 
Baltimore City Public Schools, Maryland 


Cleveland Metropolitan School District, Ohio 
Des Moines Public Schools, lowa 
Long Beach Unified School District, California 


The initiative was motivated by an effort to increase student learning and achievement by 
improving principal effectiveness. Research has shown that strong principals are integral to 
strong schools and to raising the quality of teaching. Numerous studies have pointed to the 
importance of effective leaders for teacher satisfaction, teacher retention, school climate, parent 
engagement, and student achievement. Principal supervisors are a potential point of leverage for 
supporting and developing principals, but relatively few districts have invested in such efforts. 
The motivating hypothesis of the PSI is that changing the role of principal supervisors from 
overseeing administration and operations to providing instructional leadership can drive 
improvement in principal effectiveness. 


B® The PSI includes five core components: 


Revising the principal supervisors’ job description to focus on instructional 
leadership 


1. 
2. Reducing principal supervisors’ span of control (the number of principals they 
oversee) and changing how supervisors are assigned to principals 


Training supervisors and developing their capacity to support principals 
Developing systems to identify and train new supervisors (Succession planning) 


Strengthening central office structures to support and sustain changes in the 
principal supervisor’s role 


The study of the PSI, conducted by researchers from Mathematica Policy Research and 
Vanderbilt University, will document districts’ experiences implementing the initiative and 
examine the effects of the initiative on principals’ performance. This first study report describes 
the emergence of a new role for principal supervisors in the six PSI districts, documenting 
districts’ experiences with and lessons learned from the PSI from its inception in the 2014—2015 
school year through spring 2017. A second report will describe the final year of implementation 
and examine the effects of the initiative on principals’ performance. A third report will compare 
the principal supervisors’ role in the six PSI districts with that in a national sample of urban 
districts, to learn how principal supervision in PSI districts differs from that in similar districts 
that were not part of the initiative. 
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In this report, we present analyses of data from semistructured interviews with central office 
personnel, principal supervisors, and principals, as well as data from surveys of supervisors and 
principals in each of the six PSI districts. 


A. Overview of study findings: A new role for principal supervisors emerges 


The six PSI districts demonstrated the feasibility of making substantial changes to the 
principal supervisor role, across all components of the initiative. The districts revised the job 
descriptions for principal supervisors, reduced the span of control, implemented new training 
programs, and restructured roles and responsibilities in the central office to support changes to 
the principal supervisor role. 


These changes in the principal supervisor role laid the groundwork for changes in principal 
supervisors’ day-to-day work with principals. Most principal supervisors now spend the largest 
share of their time in schools engaging in newly developed routines and practices, such as 
participating in classroom walk-throughs, coaching principals, and providing ongoing feedback. 
In some districts, they also work with assistant principals or school leadership teams. They focus 
less on administration and building operations than in the past. They also focus less on 
compliance activities, such as monitoring supplies and ensuring district and state forms are 
completed correctly and submitted on time. Principal supervisors also consistently meet with 
groups of principals to provide opportunities for collaborative learning. 


Revising the principal supervisors’ job description to focus on instructional leadershi 
£ p p p J p p 


Districts revised the job description of principal supervisors to focus heavily on 
developing instructional leadership and supporting principals. 


Through their revisions to the principal supervisors’ job description, districts articulated a new 
vision for the principal supervisor role. To inform the revised job descriptions, most districts relied 
heavily on the draft Model Principal Supervisor Standards (Council of Chief State School Officers 
2015) and sought input from a mix of central office departments, senior-level district officials, 
current principal supervisors and principals, and external technical assistance providers. ' 


Changing the job description required shifting some responsibilities previously held by 
principal supervisors to other central office staff. As of the third year of the initiative, districts 
continued to wrestle with the redistribution of responsibilities formerly held by principal 
supervisors. This highlights a challenge other districts may face as they seek to make similar 
changes to the principal supervisor role. 


| The Wallace Foundation supported the development of the Model Principal Supervisor Standards. 
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Reducing principal supervisors’ span of control and changing how supervisors are 
assigned to principals 


PSI districts reduced the number of principals each supervisor oversaw and created 
networks of principals to facilitate collaboration and small-group learning 
communities. 


Districts reduced the number of principals assigned to each supervisor, which provided 
supervisors time to focus on developing and supporting principals as instructional leaders. 
Before the PSI, across the six districts, supervisors oversaw an average of 16 principals. Within 
the first three years of the initiative, the average span of control across all six districts decreased 
to 12 principals. The number of supervisors who reported that they oversaw too many principals 
declined in every district from 2016 to 2017. 


Although most districts successfully reduced the average number of principals overseen by 
their supervisors, spans of control varied substantially among supervisors within districts. 
Districts discovered that the appropriate span might not be the same for all supervisors, 
depending on the characteristics of schools and principals in their networks. For instance, new 
principals might require more or differing supports relative to veteran principals. 


Each supervisor oversaw a network of principals. Districts grouped principals into networks 
based on a combination of grade level, geography, school theme or focus, and, at times, by 
performance level. Districts typically matched supervisors strategically to networks according to 
their relevant experience and expertise. The networks facilitated collaboration among principals, 
enabling them to share practices and participate in professional development together. 


Principals were able to spend more time interacting with supervisors because of the reduced 
spans of control and changes in network groupings. As a consequence, they reported developing 
more productive relationships with their supervisors than in the past. This change in the nature of 
the relationships between principals and supervisors was due not only to the quantity of time 
spent together, but also to the qualitative shift in the nature of their interactions, toward a greater 
focus on instructional leadership. 


In the first years of the PSI, shifting supervisor assignments and hiring new supervisors led 
to instability in relationships between supervisors and principals. As districts finalize their 
supervisor rosters, we might expect more stable principal groupings and reduced supervisor 
turnover. 


‘) Training supervisors and developing their capacity to support principals 
PSI districts developed systematic training programs to develop supervisors’ skills. 


Before the PSI, supervisors did not receive training specifically aimed at improving their 
capacity to support and develop principals as instructional leaders. By 2017, however, districts 
had demonstrated the benefits of targeted supervisor training to develop supervisors’ capacity to 
support and coach principals. The supervisor training focused heavily on understanding and 
identifying high quality instruction and developing principals as instructional leaders, although 
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the primary focus varied by district. Supervisors especially valued job-embedded training 
approaches, such as one-on-one coaching and in-school peer observations with other supervisors. 


Districts grappled with finding a balance for supervisors between spending time on training 
and spending time in schools. Although supervisor training ensures supervisors have the needed 
skills to support and develop principals, it takes away from time working directly with principals 
in schools. Districts must determine an appropriate balance between these two objectives. 


Technical assistance providers played key roles in planning and facilitating principal 
supervisor trainings. Supervisors often reported that trainings drifted from their intended 
purpose, or were limited in quality when technical assistance providers were not present. This 
suggests that technical assistance from external providers may be an important component of 
effective training. 


4) Developing systems to identify and train new supervisors (succession planning) 


Some districts inplemented apprenticeship programs to prepare promising candidates 
to become principal supervisors. 


To prepare candidates to step into the redesigned supervisor position, three districts 
developed apprenticeship programs. These programs offered a mix of mentorship and formal 
training opportunities for prospective supervisors. The programs provided participants with job- 
embedded experiences to develop skills required of principal supervisors, such as planning 
professional development for principal networks, co-planning and co-leading principal support 
meetings, and coaching a small number of principals. 


Program participants reported feeling prepared for the principal supervisor role. Districts 
also benefited as the apprenticeship programs facilitated succession planning for principal 
supervisors, offering district leaders opportunities to observe participants in action. 


Strengthening central offices to support and sustain changes in the principal 
supervisor role 


Central office departments began to coordinate more with one another, creating a 
cultural shift and leading to structural reorganization to support the new principal 
supervisor role. 


Districts worked to create new central office structures to facilitate the work between 
schools and central offices as the principal supervisors’ roles changed. Specifically, districts 
reallocated supervisors’ noninstructional responsibilities to help them focus on supporting 
principals’ instructional leadership; implemented new structures to foster collaboration and 
coordination across departments; and improved systems of communication among the central 
office, supervisors, and schools. 
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B. Looking ahead: Areas for continued focus 


The experiences of the six PSI districts demonstrate that it is possible for districts to make 
substantial, meaningful changes to the principal supervisor role. But this work is ongoing. 
Moving into the final year of the initiative and beyond, districts may focus on the following areas 
as they continue to refine the principal supervisor role and work to sustain their initial 
accomplishments. 


e Developing a common definition of instructional leadership. Some supervisors reported 
ambiguity about what it means to support and develop principals’ instructional leadership. 
In addition, the principal supervisor role in some districts has become heavily focused on 
developing high quality instruction. This can be considered a requisite element of strong 
instructional leadership, but it is only part of a multifaceted set of instructional leadership 
skills, such as developing a strong school culture and providing job-embedded professional 
development to teachers. Districts should continue to clarify the focus of the supervisor role 
to help articulate the priorities and practices for supervisors. 


e Identifying a balance between supervisors’ central office involvement and time spent 
in schools. When supervisors had to spend too much time on central office matters, they 
found they had insufficient time to visit their schools and provide the intended support for 
principals. However, too little time in the central office left supervisors out of the loop and 
disconnected from central office departments and personnel; these connections are needed to 
ensure that principals are supported. 


e Developing internal capacity to provide high quality, job-embedded training and 
support for supervisors. District leaders continue to determine how to sustain support and 
training of both new and veteran supervisors as the PSI concludes and fewer resources are 
available for technical assistance. 


e Developing and refining approaches to identify and train new supervisors. Districts that 
have developed supervisor apprenticeship programs can consider how to make time for 
participants to engage in apprenticeship programs at a high level and how to provide other 
opportunities for participants who complete the program but are not placed immediately into 
a supervisor role. 


e Continuing to shift central office departments toward a school-centered culture. This 
shift is a significant cultural change for districts. Districts will continue to work on 
consistency across departments to support principal supervisors in their new roles. 


e Maintaining existing momentum in the changes to the principal supervisor role. 
Moving forward, districts will need to work to ensure that progress continues along the 
same trajectory in the coming years, through the end of the initiative and beyond. 
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I. OVERVIEW OF THE PRINCIPAL SUPERVISOR INITIATIVE 


In 2014, The Wallace Foundation launched the Principal Supervisor Initiative (PSI), a four- 
year, $24-million-dollar effort to redefine principal supervision in six urban school districts. 
Specifically, the PSI aimed to help districts overhaul a position traditionally focused on 
administration, operations, and compliance to one dedicated to developing and supporting 
principals to improve instruction in schools. The motivating hypothesis of the PSI is that 
changing the role of principal supervisors from overseeing operations to providing instructional 
leadership can drive improvement in principal effectiveness. Improved principal effectiveness 
can, in turn, be an important lever for improving instruction and, ultimately, student 
performance. 


The Wallace Foundation also 
commissioned an independent study of the Six urban districts are participating in the PSI:2 
PSI to share lessons from the initiative with 
school districts, education practitioners, 
policymakers, and other researchers. The 
study, conducted by researchers from 
Mathematica Policy Research and Vanderbilt 
University, has two interrelated aims: to 
document districts’ experiences implementing 
the PSI and to determine its impact on 
principal effectiveness. The study will 
describe districts’ accomplishments and challenges as they implemented the PSI, highlighting 
lessons learned for other districts seeking to revise the principal supervisor role. It will also 
address the initiative’s primary question: Does shifting the role of principal supervisors in 
complex districts from overseeing compliance to sharpening principals’ instructional leadership 
capabilities improve the effectiveness of the principals with whom they work? 


Broward County Public Schools, Florida 
Baltimore City Public Schools, Maryland 


Cleveland Metropolitan School District, Ohio 
Des Moines Public Schools, lowa 

Long Beach Unified School District, California 
Minneapolis Public Schools, Minnesota 


This report, is the first of three reports from the study and traces the experiences of the PSI 
districts from the initiative’s inception in August 2014 through spring 2017 (Figure I.1). The 
report describes the districts’ individual approaches and accomplishments as well as the 
challenges they have faced. The second report will describe the continued implementation of the 
PSI in each of the six districts through spring 2018. It will also examine how principal 
supervisors differentiate support for principals and estimate the effects of the PSI on principal 
performance from the start of the initiative through spring 2018 as measured by the Vanderbilt 
Assessment of Leadership in Education (VAL-ED). The third report will compare principal 
supervision in the PSI districts with that in other urban districts throughout the country in the 
2017-2018 school year.? 


? DeKalb County School District (Georgia) was an initial participant in the PSI, but withdrew after the first year, 
following a change in district leadership. Baltimore City Public Schools joined the initiative after DeKalb withdrew. 


3 The study will not examine the PSI’s effects on student achievement because we would not expect to see these 
effects emerge during the limited time frame of the study. 
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This report is based on analyses of data from multiple sources. We conducted two rounds of 
semistructured interviews with central office personnel, principal supervisors, and principals in 
each of the six PSI districts. In summer 2015, we conducted a total of 108 interviews and, in fall 
2016, we conducted another 109 interviews. Through the interviews, we aimed to understand 
districts’ on-the-ground experiences as they changed the principal supervisor role. In addition, 
we administered surveys to all principal supervisors and principals in the PSI districts in fall 
2015 and spring 2017. The survey data provided a more general view of districts’ experiences, 
reflecting the perspectives of a broader range of supervisors and principals. We also collected 
documents and artifacts from the districts, including revised job descriptions, principal 
supervisor training agendas, and examples of protocols and tools developed to establish common 
work routines among supervisors. 


Figure I.1. Time periods covered in the three study reports 


Third report 
(national 
Current report comparisons) 


a. 


August 2014 z August 2018 
PSI begins 2014-2015 2015-2016 2016-2017 2017-2018 PS/ ends 


Me 


Second report (implementation and effects) 


A. The PSI: Background 


—The PSI evolved from The Wallace Foundation’s longstanding commitment to improving 
students’ academic achievement by strengthening the quality of educational leadership. Research 
has suggested that principal effectiveness is a significant factor in school success. For example, 
schools with effective leaders have more satisfied teachers, lower rates of teacher turnover, more 
positive learning climates, greater parent engagement, and, ultimately, higher student 
achievement (Boyd et al. 2011; Grissom et al. 2015; Grissom and Loeb 2011; Hallinger et al. 
1996; Leithwood et al. 2004; Sebastian and Allensworth 2012). Increasingly, school districts 
expect principals to improve their teachers’ performance through observations, feedback, and 
other forms of instructional leadership (Neumerski et al. forthcoming), yet districts are only 
beginning to invest in local district support for principals themselves (Goff et al. 2014). The 
principal supervisor is a natural locus for such support, and recent research suggests that 
improving the effectiveness of principal supervisors might be essential for improving the 
effectiveness of principals (Corcoran et al. 2013). 


The PSI is the outcome of a deliberate effort by The Wallace Foundation to promote a vision 
for principal supervisors as drivers of instructional leadership among principals. Researchers 
who conducted the evaluation of The Wallace Foundation’s Principal Pipeline Initiative found 
that many districts involved in the initiative decreased the supervisor-to-principal ratio for 
supervisors of novice principals to provide needed supports. In addition, these supervisors shifted 
the focus of their work with principals from administrative oversight and operations to 
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instructional leadership. A large majority of principals surveyed valued the support they received 
as aresult of this shift. Still, the same research found that the capacity of principal supervisors to 
support and coach principals varied greatly, and operational tasks stymied many supervisors’ 
ability to engage in instructional leadership support (Turnbull et al. 2016). These findings 
suggested a need to explore additional ways to leverage principal supervision to develop and 
support principals. 


B. The core components of the PSI 
The design of the PSI consists of five core components (Figure I.2), which districts 
implemented according to their local contexts and needs. 


Figure 1.2. The five core components of the PSI 


Strengthen central office 
structures to support and 
sustain changes in the 
principal supervisor role 


Goal: 


Improve Principal 
Develop systems Effectiveness 


to identify and train 
new supervisors 


Revising the principal supervisors’ job description to focus on instructional leadership. 
Principals need support from supervisors to act primarily as instructional leaders, rather than as 
building managers. Traditionally, however, central offices required principal supervisors to focus 
on compliance, such as ensuring principals had submitted appropriate forms for budgeting and 
state accountability, checking on the completion of school improvement plans, and monitoring 
whether Individualized Education Plans were up to date. This left principal supervisors with little 
time to provide support for instructional leadership. Thus, one key component of the PSI is to 
help districts reorient expectations for supervisors’ work to focus on instructional leadership in 
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schools. Revising the job description is the first step for district leaders to redefine and codify 
their vision for the principal supervisors’ role. 


Reducing principal supervisors’ span of control. It is difficult for supervisors to 
effectively support a large number of principals, especially when the role includes regular school 
visits with ongoing feedback to principals. Thus, reducing the span of control, or the number of 
principals each supervisor oversees, is a central component of the PSI. Districts are expected to 
sustain the reduced spans of control with internal funding by the end of the initiative. 


Training supervisors and developing their capacity to support principals. The 
significant shift in the revised role for supervisors requires new and different areas of expertise 
than previously required for the position. To address these needs, districts work with external 
technical assistance providers to develop supervisors’ capacity and skills to coach, mentor, and 
provide professional development for principals and to manage learning communities. 


Developing systems to identify and train new supervisors (succession planning). With 
the PSI’s support, districts identify and develop new talent to fill future principal supervisor 
positions by creating a cadre of new supervisors with the requisite capacities for the revised 
supervisor role. Approaches to this component can include apprenticeship programs to prepare 
future principal supervisors to step into the position or developing leader tracking systems to 
identify and prepare prospective principal supervisors. 


Strengthening central office structures to support and sustain changes in the principal 
supervisor role. The change in the principal supervisor role requires shifting many managerial 
tasks, which supervisors previously handled, to other central office personnel; central office 
culture and structures also change to align with and support the new supervisors’ role. 
Furthermore, as their work shifts away from administration, compliance, and operations, 
supervisors work more closely with other academics-oriented departments, such as curriculum 
and teaching and learning. Although The Wallace Foundation asked that districts create a 
detailed plan for central office change only by the conclusion of the initiative, districts began to 
address this component even in early implementation. 


C. The districts in the PSI 


The foundation chose six urban districts to implement the PSI. The foundation selected 
districts that it thought saw the potential value of redefining the principal supervisor role as a 
lever for change. At the beginning of the PSI, districts provided background demographic 
information about their student populations, number of schools and principals, and the 
corresponding span of control of the principal supervisors. 


e Baltimore City Public Schools is the fourth-largest school system in Maryland. Baltimore 
joined the PSI in August 2015, though the district had engaged in a process to redefine 
principal supervision before joining the initiative. At the start of the PSI, there were about 
104,000 students and 162 principals in the district. The average span of control for 
supervisors (now called instructional leadership executive directors) before the PSI was 13. 
The district experienced turnover in leadership in summer 2016, leading to a new 
superintendent and chief academic officer. 
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e Broward County Public Schools is the second-largest school district in Florida and the 
sixth-largest public school system in the United States. At the beginning of the PSI, the 
district served about 256,000 students and had 228 principals. Before the PSI, supervisors, 
now called cadre directors, had an average span of control of 21. 


e Cleveland Metropolitan School District is the second-largest school district in Ohio. 
When Cleveland began the PSI, the district served about 45,000 students and had 100 
principals. Supervisors, now called network support leaders, had an average span of control 
of 16 before the PSI. 


e Des Moines Public Schools is the largest district in lowa. At the beginning of the PSI, the 
district served about 33,000 students and had 65 principals. Before the PSI, the average span 
of control was 16 for supervisors, who are now called directors. 


e Long Beach Unified School District is California’s third-largest school district. There were 
about 85,000 students and 86 principals in the district at the beginning of the PSI. The 
principal supervisors in Long Beach are now called now called directors, though assistant 
superintendents supervise some principals. Before the PSI, the average span of control was 17. 


e Minneapolis Public Schools is the third-largest school district in Minnesota. At the 
beginning of the PSI, there were about 35,000 students in the district. The district employed 
57 principals. The average span of control for principal supervisors (called associate 
superintendents) before the initiative was 17. Minneapolis was without a permanent 
superintendent from December 2014 until summer 2016. 


D. The principal supervisor role before the PSI 


Across the PSI districts, central office leaders, principal supervisors, and principals who we 
interviewed in 2015 consistently described the role of the principal supervisor before the PSI as 
compliance oriented. Supervisors were heavily involved in operational issues, such as building 
maintenance, and tasked with resolving issues that schools could not handle on their own. For 
example, principals and supervisors often pointed to central office departments that were 
unresponsive to principals’ requests and would respond only when supervisors became involved. 

Some districts also expected supervisors to work on principal 
Q Before the Wallace grant, development, instruction, and school improvement. However, 
my role was anything and work on discipline, maintenance, school climate, parent 
everything, and itwas very Complaints, budget oversight, and attendance monitoring 

hard to focus on seeds bo gase 

finstrtictional Ieadeiship, regularly superseded these responsibilities. As one district 

leader put it, “every issue came to the [supervisor], so they 
Principal Supervisor (2015) — were kind of like a catch-all.” 


E. Principal supervisors early in the PSI 


Across the six PSI districts, the average principal supervisor surveyed in fall of the 2015— 
2016 school year had several years of experience in the role and was likely to have previously 
worked as a principal. On average, principal supervisors had 5.4 years of experience in their role. 
Of these supervisors, 14 percent had also served as principal supervisors in a different school 
district at some other point in their careers, spending on average 4.6 years in that position. 
Overall, however, 80 percent of 2015 supervisors had been hired internally from within the 
district. Almost two-thirds (63 percent) of the supervisors had experience as teachers. The vast 
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majority of the supervisors worked as principals immediately before taking on their current 
position (Figure I.3). 


Figure 1.3. Positions held by principal supervisors immediately before becoming 
a supervisor in a PSI district 


Principal 

supervisor in Other, 8% 
different 

district, a% 


Central office 
administration, 
12% 


Principal, 
76% 


Figure reads: 76 percent of current supervisors held the position of principal immediately before becoming principal 
supervisors. 


Source: Supervisor surveys, 2015-2016 (N = 49). 


F. Overview of this report 


In this report, we first describe the research methodology. We then describe districts’ 
experiences implementing each component of the initiative and describe the work and roles of 
principal supervisors. The final chapter summarizes how the changes districts made collectively 
contributed to the shift in principal supervisors’ work during the initial years of the initiative. We 
bring together the perspectives of central office leaders who were closely involved in the PSI, the 
supervisors themselves, and the principals with whom they work. 


In each chapter on the PSI components, we describe districts’ accomplishments and 
challenges, focusing on commonalities and variation across districts. The report highlights broad, 
cross-district themes but grounds these findings in district-specific examples to illustrate the 
importance of district context. Throughout the report, we highlight key take-away messages 
(indicated with a key icon) for other districts seeking to make similar changes to the principal 
supervisor role. 
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ll. STUDY METHODOLOGY 


The findings presented in this interim report are based on data collected by independent 
research teams at Vanderbilt University and Mathematica. This initial report focuses on the 
experiences of the six districts in implementing the Principal Supervisor Initiative (PSI) during 
the first three years, from the 2014—2015 to 2016-2017 school years. 


A. Data on implementing the PSI 


Data for this report are from site visits and surveys conducted during the 2015-2016 and 
2016-2017 school years (Figure II.1). We began data collection after the districts had already 
begun to implement the PSI. 


Figure II.1. Implementation data collection timeline 


PSI begins 2nd round of surveys 

August 2014 (March—June 2017) 
1st round of surveys 3rd round of surveys 
(Nov. 2015—Feb. 2016) (Spring 2018) 

This report 
2014-2015 2015-2016 2016-2017 2017-2018 
1st round of site visits 3rd round of surveys 
(June—Aug. 2015) (Spring 2018) 


2nd round of site visits 
(Oct.—Nov. 2016) 


Site visits and interviews. The Vanderbilt team visited each of the six PSI districts twice: 
once in the summer or fall 2015 and again in fall 2016. We conducted the first site visit about a 
year after districts had begun the initiative. Thus, we do not have baseline data collected before 
the start of the PSI, although data collection began early enough that participants often could 
draw comparisons between their current work and work before the start of the initiative. Each 
site visit lasted two to four days. During each site visit, the study team conducted semistructured 
interviews with central office staff, principal supervisors, and principals. The study team 
developed protocols for these interviews based on a review of existing research on principal 
supervision and designed the protocols to cover each of the components of the PSI. The team 
also collected artifacts, such as training agendas and supervisor job descriptions. We recorded 
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and transcribed all interviews for analysis. In total, the research team conducted 219 interviews 
across the six PSI districts (Table II.1). 


e Central office staff. The study team interviewed two central office staff members in each 
district during each site visit. Typically, we interviewed the district’s project director for the 
PSI (the main point of contact for the initiative) and the supervisor of the principal 
supervisors or someone else in the central office who was closely involved in the initiative. 


e = Principal supervisors. The study team interviewed up to six principal supervisors in each 
PSI district during each site visit. In districts with six or fewer supervisors (Cleveland, Des 
Moines, and Minneapolis), we interviewed all supervisors. In districts with more than six 
supervisors (Baltimore, Broward, and Long Beach), we selected a sample of six to 
interview. In these districts, we first stratified supervisors according to the type of schools 
supervised (for example, by grade level) to ensure we included supervisors from all types of 
schools, and then randomly sampled supervisors from these strata. For the second round of 
site visits, we reinterviewed previously sampled supervisors. If a previously sampled 
supervisor was no longer employed in the position, we randomly selected a replacement 
from the available pool of supervisors.* 


e = Principals. In each district, we selected a random sample of 10 principals, stratified by 
supervisor, to ensure that as many supervisors as possible were represented. We excluded 
principals who were new to the district in the year of the interview or were principals of 
special schools (for example, alternative night schools). In addition, we gathered 
information on principals’ school grade levels and school performance. We used this 
information to ensure that the sample of principals included a range of school grade and 
performance levels. In the second site visit, we reinterviewed 5 of the original principal 
participants in each district to capture their perspectives over time. In addition, we selected a 
sample of 5 new principals, following the procedures described earlier. 


Table II.1. Number and roles of interview participants 


Respondents per Total in second 
district in each Total in first site visit site visit 
site visit (summer 2015) (fall 2016) 
Central office staff 2-4 15 13 
Principal supervisor 6 35 37 
Principal 10 59 60 
Total 18-20 109 110 


Surveys. We conducted two rounds of principal and supervisor surveys for this report. We 
administered the first round of surveys from November 2015 to February 2016 and the second 
round from March to June 2017. During both rounds, we sent online surveys to all principals and 
supervisors in the six PSI districts. We developed the surveys based on preliminary analysis of 
interview data, a review of current supervisor competency standards, and prior research on 


4 We interviewed nine new supervisors during the second round of site visits. 
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principal supervisors. We reviewed, piloted, and revised the survey items to ensure their validity. 
Response rates for both principals and supervisors were very high across districts (Table II.2). 


Table II.2. Principal and principal supervisor survey response rates 
Round 1 (fall 2016) Round 2 (spring 2017) 


Supervisor Principal Supervisor Principal 


surveys surveys surveys surveys 


Sample Response Sample Response Sample Response Sample Response 


size rate size rate size rate size rate 
Baltimore 11 100% 122 87% 9 100% 110 80% 
Broward 10 100% 221 98% 15 100% 217 94% 
Cleveland 7 100% 97 91% 7 100% 101 90% 
Des Moines 6 100% 60 100% 6 100% 63 100% 
Long Beach 10 100% 85 97% 8 100% 85 96% 
Minneapolis 6 100% 59 92% 6 100% 64 97% 
Overall 50 100% 644 94% 51 100% 640 92% 


B. Analyses 


The research team coded the interview data using an iterative coding process, focusing on 
each component of the PSI as well as how each component unfolded and was experienced in 
each district. The data analyses focused on districts’ accomplishments over the early years of the 
initiative, the challenges districts experienced along the way, and districts’ planned approaches 
for moving forward. We used reports and documents, such as training agendas, to supplement 
the interview and survey data. Descriptive survey results are integrated with the qualitative data. 
We report unweighted survey responses, giving equal weight to the perceptions and experiences 
of respondents across the initiative. 
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lil. REVISING THE J OB DESCRIPTION 


The aim of the Principal Supervisor Initiative (PSI) is to shift principal supervision from its 
traditional focus on compliance and supervision to a focus on supporting principals and 
developing instructional leadership in schools. As part of this effort, The Wallace Foundation 
expected districts to revise the principal supervisor job description to define the qualifications 
and competencies expected from supervisor candidates and formally delineate expectations for 


the new principal supervisor role. 


A. All six districts began implementing the PSI 
by revising the job description 


Revising the job description for principal supervisors 
was the first step districts took as part of the PSI. In fact, 
before applying for the PSI, three districts—Baltimore, 
Broward, and Cleveland—had already significantly revised 
the job description to begin the shift envisioned by the PSI. 
These districts further developed supervisors’ job 
descriptions in the first year of the initiative to be more 
specific about their expectations for the principal supervisor 
role. Minneapolis began revising the job description in 
conjunction with its application for the PSI grant and 
continued to make revisions during the first year of the 
initiative. Long Beach and Des Moines prioritized revisions 
at the start of the PSI. By the end of the initiative’s first 
year, all districts reported that they considered this 
component complete, although in some cases noting that 
there could be small tweaks in the future. 


B. Revising the job description involved input 
from a variety of sources and stakeholders 


@ ...One of the pieces that we 


addressed was how the job 
description was written in 
the past was heavy on 
operations ... and very little 
on instructional leadership, 
support and development. 
And we wanted to make 
sure that when we rewrote 
the job description ... 
principal supervisors [are 
primarily] ... executive 
instructional coaches... 
Not that operational 
components of the role 
aren’t important or that you 
wouldn't ever address 
those, but that can’t be the 
primary purpose for why 
you’re showing up ina 
building. 


District Leader (2016) 


The revision process involved multiple iterations. Districts sought input from various 
sources and stakeholders, such as central office departments, senior district officials, current 
principal supervisors, and principals. As part of this process, most districts also worked with 
external technical assistance providers, such as the Center for Education Leadership, New 
Leaders, and the New York City Leadership Academy. District staff from Baltimore and Des 
Moines also reported reviewing job descriptions developed by other PSI districts. Staff from four 
of the six districts said they relied heavily on the draft version of the Model Principal Supervisor 
Professional Standards (Council of Chief State School Officers 2015) in defining the 


responsibilities of the role (Figure III.1). 
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Figure III.1. Model principal supervisor professional standards 2015 


EDUCATIONAL LEADERSHIP 


>» Dedicate their time to helping principals grow as instructional 
leaders 


» Coach and support individual principals and engage in effective 
professional learning strategies to help principals grow as 
instructional leaders. 


> Use evidence of principals’ effectiveness to determine necessary 
improvements in principals’ practice to foster a positive 
educational environment that supports the diverse cultural and 
learning needs of students. 


> Engage principals in the formal district principal evaluation 


process in ways that help them grow as instructional leaders. 


SMOOTH AND EFFECTIVE FUNCTIONING OF THE DISTRICT 


> Advocate for and inform the coherence of organizational vision, SUPPORTS TO 


policies and strategies to support schools and student learning. PRI NCIPAL 


> Assist the district in ensuring the schools with which they engage 
are culturally/socially responsive and have equitable access to 
resources necessary for the success of each student. 


IMPROVING THE CAPACITY AND EFFECTIVENESS OF THE 
PRINCIPAL SUPERVISOR AS A DISTRICT LEADER 


> Engage in their own development and continuous improvement to 


PRINCIPAL SUPERVISOR STANDARDS 


help principals grow as instructional leaders. 


> Lead strategic change that continuously elevates the 
performance of schools and sustains high-quality educational 
programs and opportunities across the district. 


Source: Adapted by the authors from the Model Principals Supervisor Standards 2015, Council of Chief State 
School Officers. Copies of the standards are available at the Council’s website 
(http:/www.ccsso.org/resource-library/model-principal-supervisor-professional-standards). 


Note: Standards fall into the three broad categories indicated in the figure. The Council of Chief State School 
Officers writes that these categories must be integrated to provide comprehensive support to principals. 


District staff typically considered revising the job description to be one of the easier tasks of 
the PSI. They involved key stakeholders in the process and relied on local and national resources 
(Table HI.1). Appendix A, Figure A.1 provides an example of a revised job description. 
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Table III.1. Timing, participants, and resources for revising the principal supervisor job description 


Timing of revisions 


Participants in revisions 


Resources for revisions 


Baltimore e Significant changes before PSI Led by chief academic officer Other districts in the PSI via New York City 
e Additional changes in 2014—2015 Input from senior-level district personnel and Leadership Academy's online professional 
a panel of principals learning community 
Broward e Significant changes before PSI Facilitated by the Center for Educational Council of Chief State School Officers’ Draft 
e Minor changes in 2014—2015 Leadership (CEL) (2014-2015) Model Standards 
Cleveland e Significant changes prior to PSI Led by chief academic officer and support Council of Chief State School Officers’ Draft 
e Additional changes in 2014—2015 team and PSI project director Model Standards 
Input from other central office department 
chiefs 
Assisted by New Leaders 
Des Moines e First PSI priority (2014-2015) Led by interim chief of schools and human Reviewed other PSI districts revisions 
resources 
Input from executive directors, central office 
personnel, and principal supervisors 
assisted by strategic leadership design 
Long Beach e First PSI priority (2014-2015) Led by deputy superintendent of schools Council of Chief State School Officers’ Draft 
Input from other central office personnel Mode! Standards 
Other related literature 
Minneapolis e Began revisions while applying for Led by Human Capital office with input chief Council of Chief State School Officers’ Draft 


PSI 
Additional changes in 2014—2015 


of schools and supervisors 


Model Standards 


Source: 2015 interviews with central office personnel. 
PSI = Principal Supervisor Initiative. 
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C. Revising the job descriptions required changing other central office roles, 
identifying a balance between central office and school-based work, and 
defining principal support 


Districts faced a number of challenges as they revised the principal supervisor job 
descriptions. 


Redistributing responsibilities to other personnel. Changing the job description for 
principal supervisors created a domino effect necessitating changes in other central office roles. 
Nearly all of the districts reported that changing the job description to focus on instructional 
leadership and support of principals marked a significant shift, requiring the districts to reallocate 
the tasks that would no longer fall under the purview of the principal supervisors. 


Identifying the appropriate balance between central office involvement and time spent 
in schools. Districts were keenly aware of the need to continue to involve principal supervisors 
in some duties not directly linked to principals and schools. For example, some districts required 
supervisors to hold formal roles on central office committees, a topic detailed later in this report. 
Districts felt that maintaining key relationships with central office staff was necessary for 
supervisors to advocate for principals and schools. Indeed, some districts were concerned that 
too much distance from the central office would lower the status of supervisors in the eyes of 
other central office personnel, reducing supervisors’ leverage when they had to advocate for their 
schools. 


Defining support for principals beyond school visits. Districts recognized that 
supervisor’s support of principals extended beyond one-on-one interactions in schools with 
principals. Supervisors often noted that principals could not focus on instructional leadership if 
operational problems consumed their energies. Districts continued to consider which types of 
tasks are central for principal support. 
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IV. REDUCING SUPERVISORS’ SPAN OF CONTROL AND ASSIGNING 
SUPERVISORS TO SCHOOLS 


One of the major components of the Principal Supervisor Initiative (PSI) is to reduce the 
span of control, or the number of principals and schools assigned to each supervisor. The goal of 
this reduction is to make supervisors’ portfolios more manageable and thus enable them to have 
more time with principals for coaching, mentoring, and instructional leadership development. 


A. Most districts have reduced the span of control 


Overall, most PSI districts reduced the average span of control over the first three years of 
the initiative (Table [V.1). The span of control across all six districts in the third year of the 
initiative (2016-2017) averaged 12 principals for each supervisor, compared with an average of 
17 before the PSI began. However, the principal-to-supervisor ratio varies substantially both 
within and across districts. Districts attributed this variation to differences in span of control at 
the start of the initiative, local context, and district policies for assigning schools to supervisors. 


Table IV.1. Average span of control and range in the PSI districts 


Pre-grant* 2015-2016** 2016-2017** 

Mean Mean Range Mean Range 
Baltimore 13 13 (9-16) 14 (12-17) 
Broward 21 21 (15-25) 15 (11-19) 
Cleveland 16 14 (14-15) 13 (11-15) 
Des Moines 16 10 (8-11) 10 (8-11) 
Long Beach 17 9 (3-12) 11 (9-13) 
Minneapolis 17 10 (7-12) 10 (6-13) 
Overall 17 13 (3-25) 12 (6-19) 

Sources: * The Wallace Foundation 2014; ** Principal supervisor surveys. 


B. Reducing the span of control was a quick and early win 


The Wallace Foundation initially expected districts to phase in reductions in principal 
supervisor spans of control over time. However, the foundation also encouraged districts to aim 
for a quick win in the PSI’s first year, and several districts set their sights on reducing the span of 
control as that initial accomplishment. Districts that achieved early reductions in the span of 
control did so by hiring additional principal supervisors and simultaneously changing the ways 
they assigned supervisors to principals. Minneapolis, for example, added three new supervisors 
in the first year of the initiative (2014—2015) while shifting from a system of regional 
superintendents with very large spans to an organization in which the district assigned schools 
and supervisors by grade level (elementary, K—8, middle school, and high school). In Des 
Moines, adding two new supervisors at the beginning of the initiative reduced the span of control 
from 16 to 10, on average. In our first round of interviews, a principal supervisor described the 
effect of this change in span: 
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This year, compared to the past, it’s night and day, which is great. In the past we 
had 19 buildings [which required] touches and problem-solving, putting out fires. 
It was not real structured time; it was just kind of flying—very reactionary. My 
role this year has really changed into being very proactive. It’s identifying 
[principals’] specific needs to increase their leadership abilities ... (2015) 


A second principal supervisor reported a similar experience resulting from a reduction in 
span: 


My span of control used to be 24, so now I'm down to about 16 schools, so I'm 
able to get more into schools and do more intentional coaching and development 
with principals, so it's absolutely allowed me to be more focused on my work and 
support in the coaching than I had in the past. (2016) 


Broward and Long Beach could not immediately 
reduce the span of control due to budget constraints, and Gre In larger districts that 
the span increased slightly in Baltimore. The relatively need to add many 
large size of the district and supervisors’ large spans at the supervisor positions, 
outset of the PSI were challenges for Broward. Central it is important to 
office staff we interviewed said that the district needed develop a shared 
more time to communicate the goals and importance of sagan me 
the change as they moved to allocate resources for these supervisor role and 
new positions. In the case of Long Beach, fiscal concerns bring all stakeholders 
meant that the district relied on principal supervisors to into the process 
serve in other administrative capacities. Early in the early on. 
initiative, seven staff members supervised principals, but 
only four were dedicated, full-time principal supervisors. 
Interviewees explained that any move to create new administrative positions could be perceived 
as increasing bloat in central office. In Baltimore, the number of supervisors increased by one, on 
average, because of staff turnover and some vacancies. 


C. Span of control reductions were largely complete by the third year of the 

initiative 

As of the third year of the PSI, the districts viewed their reductions in span of control as 
largely complete. However, several factors seem to contribute to continued wide variation in 
span of control within and across districts. First, in some districts, staff serving as principal 
supervisors also hold additional roles outside of principal supervision, and their spans of control 
are therefore smaller than those of supervisors who focus only on principal supervision. Second, 
assignment strategies that allocate supervisors based on school characteristics result in 
substantial variation in spans of control within districts. For example, Long Beach assigned two 
supervisors to the large number of middle and K-8 schools in the district and, therefore, they 
have slightly larger spans than those of supervisors at other school levels. Minneapolis, Broward, 
and Cleveland also made fewer school assignments to supervisors overseeing so-called priority 
schools, or the lowest-performing schools in the district. 
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Supervisors, principals, and central office staff universally 

Q She [my supervisor] is so praised the reductions in span of control. Central office 
enmeshed in my work. | personnel felt that principals received better support because 
ee oe they had greater access to their supervisors and more regular 
an even more intimate a 7 
understanding of the visits and contacts. Supervisors reported that a smaller span 
successes andchallenges. enabled them to understand their schools better and visit more 
It’s enabled us to have real- frequently. Many principals also noted that they saw their 
time data talks in a different principal supervisor more frequently or received more support 


way, because as the data’s _—_ after the decrease in span. 
coming out she’s here for 


us to celebrate and to talk 


. Survey data indicated that supervisors viewed their spans 
about, where we are in 


of control more favorably as the initiative progressed. In 
terms of my goals. So | : ; ; i 
Gade iit han aeeveniciel particular, fewer supervisors reported that they had insufficient 
in terms of the intimacy and _ time to visit all of their schools. In the 2015-2016 school year, 
efficacy of the coaching. 59 percent of principal supervisors agreed or strongly agreed 
HH that they did not have time to visit particular schools as often 
as needed, whereas in 2016-2017, the number of supervisors 
who agreed or strongly agreed with this statement fell to 50 percent. Surveys also showed a 
decline in the percentage of supervisors who indicated that their spans of control prevented them 
from providing principals with enough support; 32 percent of principal supervisors who 
responded to the survey in the 2015-2016 school year agreed or strongly agreed that they 
supervised too many principals to provide enough support; by 2016-2017, 16 percent agreed or 
strongly agreed with this statement. 


Principal (2016) 


D. Districts faced some challenges to reducing the span of control 


Districts faced some ongoing challenges in reducing supervisors’ span of control and 
considering how best to allocate supervisors to support and develop principals. 


Sometimes, principal supervisors serve in multiple roles. Supervisors in some districts 
continue to hold dual roles with other responsibilities in the central office, as of the third year of 
the initiative. Such arrangements were primarily, but not always, due to budget constraints. Long 
Beach made considerable efforts to reduce competing demands on supervisors by removing 
formerly hybrid central office and supervisor roles. However, even after this change, some 
supervisors retained external duties. Of the eight principal supervisors in Long Beach, for 
example, three were assistant superintendents and thus had significant responsibilities in addition 
to principal supervision. Similarly, in Broward and Baltimore, each supervisor also served as a 
formal liaison to particular central office departments, 


attending meetings and representing the perspectives of om Average span of 


the schools. control can vary for 
different types of 


Span of control varies for different types of schools. Districts 


schools. Some supervisors questioned the focus on an considered individual 
average span, indicating that the optimal span might principal and school 
depend on the specific types of schools supervised. Some needs when 
suggested that optimal spans should reflect the specific determining the span 
needs and types of principals and schools assigned to of control. 
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each supervisor, taking into account school size, student performance, and individual principal 
capacity. A principal supervisor described this dilemma: 


Twelve 300-student elementary schools is not equivalent to twelve 1,500-student 

middle schools. It’s certainly not equivalent to 12 comprehensive high schools.... 
I feel like I have a really heavy load as far as being able to support these schools, 
and that I don’t have enough time to do what they need from me, because they’re 
more complicated organizations, just purely because of size and numbers. (2016) 


Sustainability of reductions in span of control is a 


Q | think that the other hill concern. Some districts expressed concern and uncertainty 
that we have to climb now about sustaining the reduced span of control after the initiative 
that we have the structure ends. One district staff member noted a desire to explore 


in place is definitely seeing 
the results. It’s one thing to 
reduce the span of control, 
but if you’re still not getting 
the results that you need at 


reallocation of Title I or II funds or leadership funding 
opportunities allowed through the federal Every Student 
Succeeds Act. Some district officials noted that obtaining the 
district-wide buy-in necessary for allocating funding to sustain 


the schools, that’s still a reduced span of control would be difficult unless stakeholders 
disconnect. So this year is were convinced of its effectiveness. 

our baseline year with the 

span of control being down Other districts were less concerned about sustainability. A 
in our elementary schools, number of school closures over the past several years in 

to see if that’s really going = Cleveland enabled the reduction in span of control via a 


to make the difference.... 
But | think that’s going to 
be part of selling it to the 
board as well. 


reallocation of funds that were formerly earmarked for those 
schools. Minneapolis and Des Moines were not concerned 
about financially sustaining the reduced span of control, as 
they had allocated funds to cover the reduction by rearranging 
District Leader (2016) __ central office positions and responsibilities. 


E. Districts considered multiple factors in assigning supervisors to schools 


In addition to reducing supervisors’ span of control, districts considered how they assigned 
supervisors to networks of schools to develop greater coherence in the supervisors’ role and 
more logical networks of principals. The PSI did not recommend a particular approach to 
assigning supervisors to schools, and approaches varied across districts. 


Table IV.2. Supervisor primary assignment approaches to networks in the PSI 
districts, 2016-2017 school year 


Grade level Geography School theme/focus School performance 

Baltimore Vv Vv 

Broward Vv Vv Vv 
Cleveland Vv Vv 

Des Moines Vv 

Long Beach Vv 

Minneapolis Vv Vv Vv 

Sources: District interviews and principal supervisor surveys. 
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Q We have now three 


principal supervisors who 
are overseeing our 
turnaround schools. Before, 
those turnaround schools 
were mixed in with all of the 
other supervisor 
assignments. Now, they’re 
sitting under specific 
principal supervisors, 
whose span of control is 
anywhere between 10 and 
11. We did that because we 
wanted to make sure that 
we were in alignment and 
that we knew that the work 
sometimes for our 
turnaround schools and 
turnaround principals is a 
lot more in-depth than that 


Typically, districts based school assignments on one 
primary factor, such as grade levels or geography, but 
sometimes considered other factors as well. Five districts 
assigned schools to supervisors by grade level (elementary, 
middle, and high schools) (Table IV.2). Broward grouped 
schools by grade level and tried to ensure supervisors had a 
mix of high- and low-needs schools. By the third year of the 
initiative, it also began to consider geography as well, to 
reduce supervisors’ driving time. Long Beach used a multistep 
method—it first assigned supervisors to grade levels and then 
within these grade levels assigned supervisors a mix of high- 
and low-performing schools. Long Beach also assigned 
principals to supervisors in groups defined by grade levels. 
Cleveland assigned supervisors to schools based on themes, 
such as technology-focused schools or schools serving 
English language learners. One of these networks intentionally 
captured all of the highest-performing schools in the district. 
Minneapolis used multiple assignment strategies; it grouped 
all magnet schools with a single supervisor and others by 


of a non-turnaround school. 


District Leader (2016) 


grade level. Broward, Cleveland, and Minneapolis also 
created networks for low-performing schools. 


Interestingly, districts based supervisor assignments on school characteristics such as theme, 
geography (including feeder patterns), and student achievement, but rarely on principal 
characteristics. Most districts did not use principal evaluation data or other information about 
principals as a criterion for supervisor assignments. The exception was Long Beach, where some 
supervisors reported that the district took principals’ coaching needs and supervisory experience 


into account in assignments for 2016-2017. 


By hiring more supervisors and reducing their spans of 
control, districts were able to rethink principal supervisor 
assignments and start matching supervisors to schools they 
thought would benefit most from their support. Thus, districts 
also generally designed their assignment strategies to 
capitalize on supervisors’ expertise—whether in a particular 
grade level or specific school needs. Districts that assigned 
supervisors based on geography did so to reduce supervisors’ 
driving time and to allow for better coordination among 
principals in the network and across feeder schools. 


F. Districts faced some challenges in allocating 
and assigning supervisors to schools 


Districts faced a number of dilemmas in determining how 
to assign supervisors to schools and how to organize 
supervisors’ portfolios and networks. 


Q Last year I had 19 middle 


schools. | still have 19 
middle schools, but I’m not 
driving all over the district. 
So what it’s allowing me to 
do is to work in tandem 
with maybe those 1 or 2 
middle schools in that zone 
that connect to that high 
school, and then allowing 
me to work with my peer 
principal supervisor at the 
high school to make sure 
that we have a strong 
alignment of curriculum, 
and of innovative programs. 


Principal Supervisor (2016) 
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Supervisors faced multiple changes in assignments. In most districts, there was a lack of 
stability and continuity in supervisor assignments from the 2015-2016 to 2016-2017 school year 
due to (1) supervisor turnover, such as retirements or movement into other positions; (2) changes 
in supervisor assignment policy; (3) hiring of additional supervisors to reduce the span of 
control; and (4) attrition and reassignment of principals. 


In some districts supervisor assignments changed because of central office staffing needs. In 
Long Beach, for example, three supervisors called assistant superintendents also oversaw 
principal supervisors. In Baltimore, the senior instructional leadership executive director (ILED) 
position was not originally intended to supervise principals. However, due to supervisor 
tumover, Senior ILEDs often became de facto principal supervisors, directly working with 
principals in addition to fulfilling their duties overseeing other supervisors. 


Supervisor turnover and new supervisor hires influenced reassignment of supervisors to 
principals in some districts. One-third (34 percent) of principals reported experiencing a change 
in supervisor from 2014—2015 to 2016-2017, but instability in supervisor assignments varied 
across districts (Figure IV.1). Des Moines and Minneapolis had no supervisor turnover and did 
not hire new supervisors during the two-year time period. In contrast, 68 percent of the principals 
in Broward had a change in supervisor from 2015-2016 to 2016-2017. The district hired five 
new supervisors, which drove much of this turnover, and also reshuffled supervisor assignments 
during that time. 


Figure IV.1. Percentage of principals with new supervisors from 2016 to 2017 
school years 


80 - School years 2015-2016 to 2016-2017 


m4 
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Figure reads: Of principals who remained in their districts from 2015-2016 to 2016-2017, 34 percent had a different 
supervisor in the 2016-2017 school year. 


Source: Principal surveys, 2017 (N = 611). 


Note: Overall, 50 percent of these reassignments were to supervisors who were new to the district and the other 
50 percent were reassignments to continuing supervisors. 
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Changing assignments makes it more difficult to 
build trust. Shuffling assignments created difficulties. To 
best support their principals, supervisors and principals 
noted they need stability to build relationships and trust 
with each other, and for supervisors to understand the 
specific needs of the principals and schools they supervise. 
Reshuffling assignments from year to year made it more 
difficult for supervisors and principals to develop 
relationships, and for supervisors to understand school 
contexts and ascertain principals’ needs. 


Assigning supervisors to diverse sets of schools can 
create additional challenges. Central office staff and 
supervisors themselves noted the particular challenge of 
supervising a diverse set of schools. For example, in 
Cleveland, supervisors were assigned by theme, such as 
science, technology, engineering, arts, and mathematics, 
and therefore some networks included elementary, middle, 
and high schools. In other districts, supervisors might have 
only one school level, but those schools might vary widely 
on some other dimension, such as academic needs. 
Supervisors noted that both kinds of diversity led to 
challenges for differentiating supports. 


Q So by swapping the schools 


around, it means that the [new 
supervisor] doesn’t know the 
school. We went round and 
round [asking]: Is it more 
important that the supervisors 
stay with the school because 
the supervisor knows the 
school, or is it more important 
that the supervisor stay with 
the principal? We landed on 
it’s more important for the 
supervisor to stay with the 
principal. Another supervisor 
had to take on these other new 
people, and so | ended up 
having to take a couple of 
another supervisor’s 
principals.... It’s tough 
because they had a 
relationship with him and now | 
have to kind of start all over 
with them. 


Principal Supervisor (2016) 


When supervisors’ networks consisted only of low-performing schools, some principals 
complained they lacked access to higher-performing schools from which they could gain insights 
or learn new practices. As one principal asked in 2016: “If we’re all struggling with the same 
thing, where’s my exemplar?” In contrast, other principals ascribed benefits to being grouped 
with schools facing similar challenges, such as conversations more focused on a common set of 


issues. 


ee) My schools are a big range of performance. Some of the lowest-performing 
schools in the district are with me ... some of the higher-performing in the 
district are with me, too. So because of that, the support that | have to provide 
is not one size fits all.... If | were to have more resources ... | could do more 
for [certain] schools without neglecting the other ones, because that’s kind of 
what tends to happen. You put your focus on the people that have the most 


requirements and the most needs. 


Principal Supervisor (2016) 
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V. TRAINING AND DEVELOPING CAPACITY OF PRINCIPAL SUPERVISORS 


A key component of the Principal Supervisor Initiative (PSI) is to provide specific and 
dedicated training to support principal supervisors as they take on a revised role. Before the 
initiative, supervisors typically received training alongside principals or other central office 
staff, and the training focused on rolling out new district initiatives. Training sought to ensure 
that supervisors were well versed in what their principals were asked to do, rather than 
developing supervisors’ capacity to develop and support principals’ instructional leadership. 
Implementing the PSI necessitated revising the content and delivery of supervisor training to 
prepare supervisors for the new expectations for the role. 


By the 2016-2017 school year, all districts had 
implemented training opportunities specifically aimed at 
developing the role-specific capacity of their principal 


@ ... [H]ow do we train them 
to actually do that jobina 
different way than what 


supervisors (Table V.1). Survey results indicate that they’ve always done? ... 

principal supervisors participated in an increasing amount Even if you put something 
of professional development specifically dedicated to their new on paper and show it 
role. In fall 2015, 61 percent of supervisors reported to somebody, that doesn’t 


mean that they’re 
necessarily going to just 
follow the new position 
description. 


participating in role-specific training, but by spring 2017, 
80 percent reported participating in such opportunities. 


In three districts, central office personnel also 
participated in supervisor-dedicated training. These District Leader (2015) 
districts included central office personnel for two primary 
reasons: to increase communication and capacity across departments and to build internal 
capacity to continue training after the initiative ceased to provide funding for external 
technical assistance providers. 
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Table V.1. Timeline and content of principal supervisors’ training, by district 


Long Beach 


Minneapolis 


Source: 


2014-2015 school year 


Planning 
No systematic training program in place 


External conferences reported as main source of 


training 


Supervisors engaged in a coherent training 
program that met regularly and relied heavily on 
technical assistance providers 

Primarily classroom-based with focus on 
understanding high quality instruction, alignment 
with instructional framework 


Supervisors engaged in a coherent training 
program that met regularly led by technical 
assistance providers; training with strong district 
input 

Heavy focus on developing instructional 
leadership and implementing new principal 
evaluation system with fidelity, developing 
coaching skills 

Classroom trainings and site-based 

1-on-1 coaching 


Planning 
No systematic training program in place 


Intermittent meetings focused on team building 
to develop supervisor cohesion 


Interviews with central office staff and training agendas. 


2015-2016 school year 


Internally led trainings with high quality instruction focus, 
norming around student learning objectives 

Midyear supervisors began consistent work with technical 
assistance provider 

Some site-based 1-on-1 coaching focused on implementing 
technical assistance provider trainings 


Technical assistance provider-led training occurred regularly 
with strong focus on high quality instruction 

Classroom-based training focused on high quality instruction 
with site-based, 1-on-1 coaching to apply classroom knowledge 
and practices 

Optional technical assistance provider-led site-based training 
on identifying high quality instruction 


Continuation of 2014—2015 training 

Deeper engagement in recognizing and calibrating high quality 
instruction 

Occasional technical assistance provider-led 

1-on-1 coaching 

Mix of classroom- and site-based training 


Monthly daylong collaboration and planning meetings covering 
supervisor goal-setting and evaluation development, 
collaborative sharing of knowledge and problem solving, data 
use training, and lab day reflections 

Site-based lab days for supervisor skill building with on-the-spot 
feedback and coaching 

Technical assistance provider-led, site-based 

1-on-1 coaching 


Regular technical assistance provider-led, high quality, 
instruction-focused training with emphasis on learning walks, 
standards, data use, developing routines, and calibrating 
supervisor practice 


Infrequent meetings to share practices and ideas 


2016-2017 school year 


Planning to implement technical assistance provider-led, instructional 
leadership-focused classroom and site-based training 

Weekly supervisor meetings to plan principal meetings with Curriculum 
Office 

Intermittent coaching check-ins with superiors for personalized skill 
development 


Focus on implementing 2015-2016 classroom learning with deliberate 
shift to site-based, 1-on-1 coaching with technical assistance provider 
Weekly collaborative school visits with 2 or 3 supervisors to practice 
skills and routines and to calibrate practice for consistency (occasionally 
attended by technical assistance provider coach) 

Monday meetings are a forum for feedback from school visits with 
dedicated time for disseminating information 


Strong focus on high quality instruction 

Most trainings are technical assistance provider-led, site-based, and 
focused on identifying high quality instruction, coaching principals to 
drive rigorous instruction, and leadership walks 

Technical assistance provider-led, site-based observation and coaching 
of supervisors based on work with principals 


Monthly collaboration and planning meetings are a mix of planning, 
problem solving, and training 


Focus on piloting supervisor evaluation, data use, curriculum 
implementation, norming practices, coaching skills, providing verbal and 
written feedback 


Mixed internal and technical assistance provider-led 1-on-1 site-based 
coaching 

Monthly executive coaching via phone on managing the demands of the 
job 

Lab days 


Technical assistance provider-led classroom-based training focused on 
developing coaching skill and work routines 


Regular meetings with department head at times devoted to training 


Site-based and remote technical assistance provider-led 1-on-1 
coaching 
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A. Training focused on high quality instruction and developing instructional 
leadership 


In supervisor training conducted during the PSI, districts typically prioritized either high 
quality instruction or developing instructional leadership. District priorities for the content of the 
training varied by the district’s vision of the role of the supervisor and of the necessary skills for 
supervisor success. Some districts focused training on instruction because supervisors did not 
have a common understanding of high quality instruction, and the districts viewed the capacity to 
recognize high quality instruction as a precursor to instructional leadership. Other districts could 
place stronger emphasis on developing instructional leadership because supervisors already had a 
common understanding of high quality instruction and language for discussing instruction, which 
provided a baseline for focusing on instructional leadership. 


Training on areas of high quality instruction typically involved reaching a common 
understanding about standards, calibrating observations, reaching agreement about protocols for 
learning walks, establishing common routines for gathering information about what happened in 
classrooms, and developing and using tools to help develop these practices. In one district, 
supervisors worked together to align instructional content to standards, engaging in the actual 
work done by students—for example, as one respondent described, “doing the math the kids do.” 
In a 2016 interview, a leader in that district explained that the goal of training was “really to try 
to give [principal supervisors] a lot of exposure and some tools so that they feel more 
comfortable talking about the areas that are not within their expertise.” 


Given their typical past experience as teachers and principals, supervisors might be expected 
to readily recognize high quality instruction. However, some districts only recently adopted new 
protocols and frameworks for defining high quality instruction that were not familiar to 
supervisors from their time as principals. Des Moines, for example, implemented an approach 
called Schools for Rigor in some schools, providing intensive training and resources focused on 
increasing the rigor of students’ academic work. In these schools principal supervisors noted that 
they had to relearn how to observe the classrooms. In other cases, principals and supervisors 
recognized their limited ability to identify whether teachers were teaching appropriate content at 
the appropriate grade level in the context of the Common Core State Standards. This was 
especially true for core subjects such as mathematics and literacy. In Baltimore, supervisors 
realized they had to spend some time familiarizing themselves with the math content before they 
could assess whether teachers were teaching at grade level. In Broward, a supervisor described 
attending numerous literacy conferences to help him identify grade-appropriate instruction in 
reading. Thus, although individual supervisors may recognize high-quality instruction, 
developing a shared, district-wide understanding remains an ongoing challenge. 


Training to develop instructional leadership focused primarily on ensuring supervisors had 
the skills to effectively communicate and develop principals, particularly through providing 
feedback and coaching. Some districts also dedicated some training to delivering feedback 
within the principal evaluation cycle, developing areas of teacher professional development and 
engaging with assistant principals and leadership teams. Training on developing instructional 
leadership also included training in data access and analysis. 
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Table V.2 lists the training topics emphasized the most and the least in the 2016-2017 
school year, across all six PSI districts. 


Table V.2. Areas of emphasis in principal supervisor training, 2016-2017 
school year 


Five most emphasized areas Five least emphasized areas 


Observing classrooms to identify instructional quality Developing principal professional learning communities 
and/or other collaborative principal groups 


Determining protocols and procedures for school walk- Developing growth plans for principals 
throughs 


Improving student growth and achievement Conducting difficult conversations 

Skills for coaching principals Using principal evaluation data to identify areas for 
improvement 

Coaching principals on giving teachers actionable Supporting principals in planning and conducting faculty 

feedback meetings and/or trainings 


Source: Supervisor surveys, 2017 (N = 50). 


Note: The survey question read: Thinking about the professional development for principal supervisors you 
attended during the 2016-2017 school year and the summer before, how much emphasis was placed on 
the following areas? 


B. Districts implemented both job-embedded and non-job-embedded 
structures of training 


Training opportunities for principal supervisors 


implemented as part of the PSI typically included non- : 
Gu For the first time, 


most supervisors 
received systematic, 


job-embedded approaches such as conferences and 
classroom-based meetings, as well as job-embedded 
approaches such as one-on-one coaching and peer 


: ; ene . job-specific 
observations. Over time, most districts increased job- professional 
embedded training opportunities for supervisors. development aligned 
Supervisors consistently reported that these job- with the new role. 


embedded opportunities were more transformative; they 
appreciated receiving real-time feedback and applying 
tools and skills introduced in non-job-embedded settings. 
(See Appendix A, Table A.2 for more details.) 
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Non-job-embedded approaches 


Group meetings. All of the districts held classroom-based group meetings for training, 
usually weekly or monthly. Early on, these sessions relied heavily on outside technical 
assistance providers. In some districts, the meetings progressively shifted from relying on 
technical assistance providers to facilitation by central office personnel. Some examples of 
the focus of group- and classroom-based training included role-playing coaching 
conversations, developing new tools for school visits, learning to use tools offered by the 
technical assistance providers, studying state academic standards for students, and studying 
the principal evaluation tool to ensure consistency across supervisors. The supervisors often 
credited this classroom-based learning, particularly when led by a technical assistance 
provider, with contributing to their growth. For instance, supervisors who learned a 
systematic approach to coaching believed it improved their work with principals. 
Supervisors who studied state standards reported they could better evaluate quality of 
instruction during classroom visits. 


Conferences. Conference attendance was the primary training opportunity available to 
supervisors before the PSI, but it receded in importance as the districts implemented 
supervisor-specific training and development opportunities. Supervisors described 
conferences as informative and viewed them as a good opportunity for networking with 
supervisors from other districts. They also appreciated the opportunity to choose topics 
aligned with their own perceived needs. 


Job-embedded approaches 


One-on-one coaching. All of the districts developed 

training opportunities for principal supervisors to G- 

receive direct observation, feedback, and coaching on One-on-one coaching 
their work with principals in schools. Technical and peer observations 
assistance providers delivered the vast majority of aoe key strategies for 
this training. Training that focused on high quality Salning Banca 
instruction often included opportunities to conduct si piahae ce 

learning walks for assessing instruction in real time, 

and practice providing feedback to principals. In 

districts that focused training on developing instructional leadership, coaching addressed 
how a supervisor might help principals create professional development opportunities and 
supports for teachers, and subsequently assess the use of those resources in classrooms. 
Supervisors reported that one-on-one support enabled them to implement new practices and 
receive personalized feedback and individual coaching. 


Peer observation. Some districts incorporated forums for principal supervisors to observe 
one another in schools and provide feedback. In these districts, supervisors conducted group 
observations of one of their peers conducting a typical school visit, including a building 
walk with classroom observations and interacting with the principal. Supervisors then 
provided feedback or identified effective practices to adopt. For example, following 
implementation of the PSI, Long Beach added Lab Days, a professional development 
structure for supervisors to practice their instructional coaching skills, delve into the work of 
supervision in the day to day context of the schools, and align their practices with one 
another. 
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C. The role of technical assistance providers was central to developing 
quality-training approaches 


The PSI specifically allocated funding to districts to hire external technical assistance 
providers to use research-based curricula to help develop and provide training for principal 
supervisors. Technical assistance providers played a central role in planning and leading training 
sessions, as well as serving as one-on-one coaches for supervisors. In addition to providing 
expertise, technical assistance providers ensured that time was reserved for training. In almost all 
districts, technical assistance providers visited schools alongside principal supervisors, observed 
their practice, provided feedback, and coached them on 
ways to improve their performance. Technical 


assistance providers also helped supervisors develop re Outside tochiical 
routines, build baseline knowledge around instructional assistance providers 


content and standards, and implement coaching played key roles in 
approaches. In addition, technical assistance providers building the supervisor 
encouraged supervisors to think about ways to professional 

document their work with each principal and develop development programs 
mechanisms for creating coherence from one visit to within the districts. 


the next. Overall, supervisors indicated that technical 
assistance providers added significant value to the 
training they received. 


By the 2016-2017 school year, three districts decreased their reliance on technical assistance 
providers by shifting facilitation of training to central office personnel. Districts cited cultural 
and financial reasons for this shift. For example, in contrast to some districts that consistently 
relied on external organizations, Long Beach had rarely relied on technical assistance providers 
before the PSI and continued to view them as a short-term, initial source of expertise that could 
then be developed internally. Districts were also aware that heavy reliance on technical 
assistance providers was not financially sustainable and sought to develop internal capacity to 
continue the training when grant funding ended. The other three districts remained almost 
entirely dependent on outside expertise for training, but each had begun to develop internal 
practices that could reduce their reliance on technical assistance providers. All three aimed to 
provide training in-house, and envisioned that either the principal supervisors’ department head 
or direct superiors would take over one-on-one coaching. 


Supervisors often reported that trainings drifted from their intended purpose or were not of 
the same quality when technical assistance providers were not present. 


D. Principal supervisors were generally satisfied with their training 


Supervisors generally reported positive perceptions of the training they received and felt the 
skills they learned improved their ability to meet the needs of their principals. Supervisors most 
appreciated time spent in schools with technical assistance providers who modeled practices, 
observed supervisors with principals, and provided immediate feedback and coaching. 
Occasionally, they viewed classroom-based meetings as disconnected from the daily work of 
supervisors, and some supervisors expressed a tension that though they acquired skills and 
knowledge in these meetings, the time dedicated to them was also time that could be spent in 
schools. 
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On surveys, supervisors assessed the quality of their training on 19 different aspects. 
Supervisors tended to agree or strongly agree that their training was engaging and useful. Most 
supervisors indicated that the training opportunities taught them new knowledge and skills (86 
percent agreed or strongly agreed), specifically enhanced their capacity to develop principals’ 
instructional leadership (83 percent), and were engaging (81 percent) and interactive (83 
percent). On average, supervisors’ ratings of training quality across all 19 aspects averaged 3.7 
ona 5.0 scale (Figure V.1).° 


Figure V.1. Supervisors’ perceptions of training quality, 2017 
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Figure reads: Principal supervisors in Broward assessed the quality of their training to be an average of 3.8 out of 5, 
on a five-point Likert scale of agreement. 


Source: Supervisor surveys, 2017 (N = 50). 


Note: Scale is composed of items measuring the following constructs: alignment with the PSI’s goals, job- 
embeddedness, active/engaging, and overall usefulness. The survey question read: “Thinking about the 
professional development and training for principal supervisors you attended during the 2016-2017 
school year and the summer before, to what extent do you agree or disagree with each of the following 
statements?” (See Appendix A, Table A.1 for details.) 


E. Districts faced some challenges in implementing supervisor training 
programs 


The challenges to improving supervisors’ capacity and training centered largely on 
determining the right balance of program content, making time for trainings, determining who to 
involve in trainings, and sustaining the amount and quality of training when the presence of 
technical assistance providers decreased. 


Some districts almost exclusively emphasized training on high quality instruction over 
developing instructional leadership. In some districts, training had greater emphasis on 
monitoring and assessing high quality instruction, rather than developing principals as 
instructional leaders. In these instances, supervisors rarely articulated a through-line connecting 
their own understanding of high quality instruction to developing their principals as instructional 


° See Appendix A, Table A.1 for information on the construction of this scale. Districts provided limited dedicated 
training to supervisors in the early year of the initiative; we do not have sufficient data to report the quality of 
training before the 2017 survey because so few supervisors received dedicated training before 2017. 
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leaders of schools. For instance, principals and supervisors described visits in which they 
observed classrooms together and then debriefed afterward. The debriefing typically focused on 
assessing the number of classrooms where they observed rigorous learning, or reaching 
agreement about their assessments of the quality of instruction, but the conversation did not 
extend to strategies by which the principal could drive instructional improvement by providing 
feedback, professional development, or other supports to teachers. 


Classroom-based training often lacks practical application. Principal supervisors in some 
districts described classroom-based learning as impractical or detached from the actual work of the 
supervisors. Principal supervisors wanted their classroom-based training to link specifically to 
practice in schools. In 2016, district leaders we interviewed said that supervisors appreciated 
deliberate attempts to conduct training “in the school, in the work,” and to provide supervisors with 
“feedback on the spot, instead of this separate learning in isolation.” They noted that this type of 
training helped show supervisors how the learning is “embedded in the day-to-day.” 


It is important to consider carefully the goals for including other central office 
personnel in supervisor trainings. Sometimes central office staff also participated in supervisor 
trainings. Central office leaders explained that involving personnel from other central office 
departments in supervisor training sessions offered benefits such as breaking down silos between 
offices, increasing understanding of the work of principal supervisors, helping with 
sustainability, and providing a forum for sharing knowledge and expertise. However, at times, 
the presence of central office personnel shifted the focus from principal supervisors’ 
development to other areas, such as instructional quality. For instance, one district found that 
including personnel from the curriculum department in supervisor trainings resulted in an 
overemphasis on program implementation and compliance in discussions. 


Time is a valuable commodity. Supervisors reported feeling an increasing urgency to be in 
their schools as often as possible. They acknowledged the importance of training but often noted 
concern that training, especially classroom-based training, encroached on their time in schools. 


Supervisors want opportunities to share practice. Some supervisors desired opportunities to 
convene collectively outside of a standard training. Many supervisors independently developed 
tools and systems to organize and implement their work, which they were never able to share. 
Supervisors spoke of the wide range of strengths and expertise their colleagues possessed, and noted 
the limited opportunities to capitalize on and share this collective knowledge. 


When technical assistance providers reduce their involvement, dedicated time for 
training and perceived quality declined. When no longer facilitated by technical assistance 
providers, group training meetings and internally facilitated classroom-based training reportedly 
drifted from structured learning opportunities toward informational sessions or school 
monitoring. In one district, principal supervisors unanimously reported a tendency for 
professional learning to become bogged down with “informational stuff.” When responsibility 
for coaching shifted to district staff and was not supported by technical assistance providers, 
supervisors felt coaching was not nearly as effective because it did not occur regularly or 
because district-based coaches were perceived to lack capacity. This finding suggests a potential 
area of concern for PSI because one-on-one coaching is the primary mechanism for 
differentiating training for supervisors. 
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VI. THE ROLE AND WORK OF THE PRINCIPAL SUPERVISOR 


The theory of action for the Principal Supervisor Initiative (PSI) hypothesizes that changes 
to the day-to-day practices of principal supervisors drive improvements in their effectiveness. 
Before the PSI, districts provided little direction for supervisor work outside of formal evaluation 
and compliance responsibilities. By the third year of the PSI, new expectations for the role (as 
articulated in the revised job descriptions), reorganized networks with a reduced span of control, 
and new approaches to supervisor training facilitated important changes to the work of principal 
supervisors in the PSI districts. 


A. Principal supervisors focused their work on principal support 


By the third year of the PSI, the focus of the redesigned Q 
principal supervisor role was robust and well understood in hea serene kind 
all six districts. District leaders expressed a consistent scone amines 


p hati ieerah aeial : : umbrella or maybe catch all 
perspective that the role of the principal Supervisor 1S to the extra things going on to 


focus on instructional leadership, specifically through help support principals so 
coaching and supporting principals, and that principal they could focus on what 
supervisors should spend most of their time in schools. They they needed to ... and we 
consistently described operational or managerial aspects of were really kind of boots- 


on-the-ground going out to 
buildings and working with 
principals as well, but 
spread very, very, very, 
very thin. 


the role, such as liaising with the central office and 
addressing parents’ complaints, as secondary to a focus on 
principal support and instructional leadership. 


Supervisors described the new role as more proactive 
than reactive. In the past, the principal supervisor simply Principal Supervisor (2015) 
responded to every need a principal voiced. By the third year 
of the initiative, leaders in each PSI district viewed the supervisor role as one with a specific 
purpose, direction, and goal: to support principals to improve schools. 


Within this common understanding across districts of an overall role focused on principal 
support, each district had a unique focus that articulated the specific orientation of the 
supervisors’ work with principals. Supervisors in some districts focused their work more 
exclusively on instruction itself, working with the principal to identify and reach agreement 
about instructional quality. For example, one supervisor described consistently working with 
principals to identify the appropriate grade level of observed mathematics instruction. In other 
districts, the supervisors’ work focused more broadly on developing leadership, supporting 
principals in taking steps to improve as school leaders or other areas not exclusive to identifying 
instructional quality. For example, one leadership development-oriented supervisor described 
most of his work with principals as focusing on the systems and programs principals used to 
address particular instructional needs. Although these two foci were not mutually exclusive, the 
distinction is evident in terms of supervisor work and emphasis in each district. Supervisors in 
Long Beach, for example, tended to focus on developing leadership, but a clear conception of 
high quality instruction informed their work with principals. Alternatively, several Baltimore 
supervisors focused most of their attention on high quality instruction, but also described 
examples of principal development more broadly. 
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B. Principal supervisors spent most of their time on instructional leadership 


Central office staff, supervisors, and principals 

across districts emphasized that principal supervisors 2 

Liar Supervisors spent 
spent much of their time in schools, focused on sue 
. : ; : : about half of their time 
instructional matters and working directly with Ee 

a - visiting schools. 

principals. Supervisors themselves also reported 
spending most of their time (63 percent) working directly 
with principals in schools or network group meetings in 
2017 (Figure VI.1). On average, supervisors spent 49 percent of their time visiting schools 
during a typical week over a three-month period in the third year of the PSI and another 14 
percent of their time with principals in networks or groups. Central office meetings (19 percent) 
and meetings with other principal supervisors (14 percent) accounted for the rest of their time, 
along with other tasks (4 percent). 


Although the average time supervisors reported spending on school visits increased only 
modestly (from 45 percent in 2015 to 49 percent in 2017, on average), interviewees distinguished 
how they used school visit time as they implemented the PSI.° Specifically, they reported an 
increased emphasis on targeted walk-throughs, principal coaching, and developing coherence 
across visits after implementing the PSI. Still, supervisors’ time varied substantially, even within 
districts. For example, supervisors’ reported percentage of time spent visiting schools in a typical 
week ranged from 0 to 90 percent in 2017. 


Figure VI.1. Supervisors’ use of time, 2017 
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Figure reads: On average, supervisors spent 49 percent of their time visiting schools in 2017. 
Source: Supervisor surveys, 2017 (N = 50). 


Note: The survey question read: “Over the past three months, what percentage of time did you spend on each of 
the following activities in a typical week, excluding travel time?” 


® The 2015 data are based on principal supervisors’ survey responses on the 2016 survey, which asked supervisors 
to recall their use of time the previous year. 
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As a Strategy for increasing school-focused use of 


time, some PSI districts gave specific targets to their Qrew Some districts give 
supervisors for time in schools. Des Moines principal supervisors 
recommended that supervisors spend 70 percent of their specific targets for 
time in schools with principals, and Minneapolis time spentin schools. 


stipulated that supervisors spend four days a week in 
schools with principals. 


Supervisors in every district had some responsibilities that required them to participate in 
central office meetings. The districts strategically scheduled such meetings to provide principal 
supervisors with substantial blocks of time to work in school buildings. For example, Des 
Moines only allowed central office departments to schedule meetings on a certain day of the 
week. Broward similarly instituted a policy that central office meetings must be scheduled after 
3:00 p.m. to protect supervisors’ time to visit schools. 


Frequency of school visits varied. In 2017, principals reported meeting with their 
supervisors at school an average of about four times in the past three months, similar to what 
they reported in 2016. However, the frequency of visits to particular schools varied significantly 
(Figure VI.2). Some principals reported seeing their supervisors as many as 20 times in a three- 
month period, whereas others did not see their supervisors at school at all. Supervisors frequently 
reported that a particular school or handful of schools required most of their time. 


Figure VI.2. Principal reports of the number of meetings with their supervisor at 
school in 2017 during a three-month period 
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Figure reads: “In 2017, principals reported meeting with their supervisor at school an average of about four times in 
the past three months.” 


Source: Principal surveys, 2017 (N = 611). 


Note: The survey question read: “Over the past three months, how many times have you met with your principal 
supervisor in the following settings? In your school.” 
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Some districts established minimum expectations for how often principals should expect to 
see their supervisors at school. For example, Long Beach required supervisors to visit each 
school at least every other week, with supervisors’ discretion to determine additional visits. The 
remaining districts provided supervisors with full autonomy over the frequency of school visits. 


Both principals and supervisors emphasized that they remained in close contact outside of 
school visits. Principals indicated that their supervisors were available to talk on the phone and 
were responsive via text or email. Principals who reported rarely seeing their supervisors often 
reported having little need for more frequent visits and, as one principal noted, “If I said that I 
needed her, she would come right over” (2016). 


The focus of the supervisors’ work related largely to instructional leadership. Principals 
reported that their supervisors spent more time on instructional leadership matters than on 
operational issues (Figure VI.3). On average, principals noted that supervisors spent more than 
half of their time working with them on instructional leadership during a three-month period in 
the third year of the initiative. 


Figure VI.3. Supervisors’ time spent in a typical week working with principals on 
instructional leadership and operations in the PSI districts, 2017 
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Figure reads, “Long Beach supervisors spent 65 percent of their time working with principals on instructional 
leadership in a typical week in 2017.” 


Source: Principal surveys, 2017 (N = 611). 


Note: The survey question read: “Over the past three months, what percentage of time did you spend working 
with your principal supervisor on each of the following?” Time on instructional leadership decreased from an 
average of 62 percent in 2016 to 54 percent in 2017, and time on operations increased slightly on average, 
from 16 to 19 percent in the same time period. We attribute this to supervisors qualitatively using the time in 
schools differently and a deeper understanding of instructional leadership practices. 
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However, districts found that ignoring operational OP icin fehitd began, the 


matters completely was often counterproductive. push was so heavily on 
Supervisors and principals reported instances when instructional leadership, 
supervisors had to balance instructional leadership support organizational management 
with support for operations-related challenges facing was ignored to the point 
principals. Supervisors noted that ignoring principals’ that it became detrimental 
operational challenges tended to worsen and ultimately : 

distract from instructional leadership. District Leader (2016) 


Operations, however, was only one of many matters that vied for the time supervisors and 
principals spent together. In addition to relying heavily on their supervisors for support related to 
instructional quality, principals also reported several other areas for which supervisors were their 
main contact, such as budgeting and curriculum and personnel issues (Figure V1.4). 


Figure VI.4. Principals’ reports of areas in which supervisors were main contact 
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Figure reads: “70 percent of principals agreed that their supervisor was their main contact for instructional quality 
issues in 2017.” 

Source: Principal surveys, 2017 (N = 611). 

Note: The survey question read: “Thinking about the past three months, to what extent do you agree or disagree 


that your principal supervisor is your main contact for support on the following?” The data from the 2016 
surveys are similar; one exception is budgeting issues, with a decrease of 12 percent from 2016 to 2017. 
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Supervisors also worked with school personnel : 

other than principals. Supervisors sometimes worked Or Despite a focus on 
with other personnel, such as assistant principals, when ee 2 

ae ae leadership, supervisors 
visiting a school. For example, Broward principals i : 

. . retained an important 

reported that supervisors regularly included the full operational and 
school leadership team in building walk-throughs and managerial role with 
school-based meetings. About one-quarter of principals their principals. 
in other PSI districts reported similarly that supervisors 
worked directly with assistant principals, coaches, and other school leaders at least some of the 
time. Direct work with teachers was rare, but when present it included activities such as visiting 
teachers’ professional learning community (PLC) meetings to provide feedback. 


C. Principal supervisors developed principals’ instructional leadership 
through walk-throughs, feedback, and coaching 


Supervisors worked with principals to develop their instructional leadership through walk- 
throughs, feedback, coaching, leading principal networks and learning communities, and 
evaluating principals. 


Walk-throughs. Walk-throughs were increasingly central to the interactions between 
principals and supervisors during the early years of the initiative, providing a platform for 
feedback and coaching. A walk-through generally included joint observations of instruction 
followed by a debriefing. 


In all but one district, respondents reported 


e Des Moines supervisors used the that, before the PSI, walk-throughs were 
iObservation observation management tool commonly a mechanism to investigate 
HOMIE LSU Mele Tek compliance. Walk-throughs addressed whether 
Long Beach’s collaborative inquiry visits instruction aligned with the district’s 
included multiple supervisors and central instructional framework. Most supervisors and 


pice member Wa elec Coucience aid principals indicated that although this alignment 


was still discussed after the PSI was 

Cleveland formalized group walk-throughs implemented, walk-throughs also expanded to 
cule) TEN Sale MSU se eNO. 0eENe include questions about feedback the principal 
central office support team to ensure team . 

MEMIbEre Nonbaccdun therechiacl would provide to the observed teacher, next 
understood school needs. steps with the leadership team, and potential 
professional development or other scenarios to 


consider. 


shared learning across the district. 


Nonetheless, some supervisors reportedly approached walk-throughs as an approach to 
oversight rather than principal coaching, and their principals similarly viewed walk-throughs as 
an exercise in compliance. For instance, principals and supervisors in one district described the 
goal of walk-throughs as a way to hold principals accountable for the instruction in their 
building, with little follow-up or support by the supervisor. 
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. Feedback. Principals said that before the PSI the Se ee 
primary feedback they received from their supervisors Gu 


sae : supervisors worked to 
occurred through the principal evaluation process. In improve instructional 


contrast, they described the feedback they received leadership through 
during the PSI as ongoing and they received it in multiple activities such as 
settings and in response to a range of principal activities. classroom 

They also perceived this regular feedback from principal walkthroughs, 
supervisors as helpful to their work. In 2017, three- systematic one-on-one 
fourths (76 percent) of principals surveyed reported that principal coaching, 
their supervisors usually or always provided them with and principal : 
actionable feedback (Figure V1.5); responses were similar professional learning 


in 2016. communities. 


Supervisors described walk-throughs as a primary means to offer principals feedback. 
Principals and their supervisors often observed classrooms jointly, followed by debriefs that 
served as an opportunity for supervisors to provide feedback to principals. In this setting, 
supervisors provided feedback on principals’ own feedback to teachers about the observed 

instruction, aptly described by a Minneapolis supervisor as 


ee) Because it’s [ongoing “feedback on feedback.” Supervisors also offered principals 
feedback] more frequent. It's | feedback on data analysis activities, teacher learning 
not, three times a year. It's community meetings, and professional development. 


detailed, it’s something that’s 
happening right then and 


More than 70 percent of principals reported that their 
there, and it’s not a standard P P P P 


What youre uae Ine ABOU supervisors usually or always documented what they discussed 
So definitely, the week-to- during a school visit in 2017, similar to 2016. For example, 
week feedback is [helpful]. supervisors commonly sent notes to the principal via email or 
other shared electronic resource following a visit, documenting 
the details of the visit or providing feedback. 


Principal (2016) 


One Long Beach principal described the way her supervisor used an online document 
sharing tool to promote coherence in support: 


We have a shared file in Google, and so ... I shared my action plans with her. We 
share notes back and forth in Google.... If she visits a meeting ... she provides 
feedback that way.... We revisit the goals and ... we talk about how things are 
progressing in my action plan. (2016) 


Fewer principals (51 percent in 2017 and 53 percent in 2016) reported that their supervisors 
usually or always used a system to monitor growth from one visit to the next. 
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Figure VI.5. Principals’ perceptions of approaches to supervisor feedback, 2017 
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Figure reads: 31 percent of principals reported that their supervisors usually and 45 percent of principals reported that 
their supervisors always provided them with actionable feedback in 2017. 


Source: Principal surveys, 2017 (N = 611). 


Note: The survey question read: “Over the past three months, thinking about all the time you spent with your 
principal supervisor, how often were each of the following true?” 


Coaching. Supervisors and principals used the term 
coaching to describe much of the work they did together. Q Sometimes I'll say, “Gosh 
Principals described coaching as a process to support them darn it, give me the answer. 
through inquiry rather than directives. Supervisors described laeausinmpalpapan™ 


Ba ; . And he never does. And 
coaching in terms of conversations about observing some-atthe otter princigals 


instruction, providing feedback about instruction, responding in our portfolio would joke 
to instructional issues in the building, using data, and other around with that, but he 
problems of practice. truly has the coaching 
model down, | would say as 
To ensure consistency and alignment with goals, some a supervisor, because he 
districts had taken steps to define a coaching model and always coached me to a 


decision.... He’ll never, ever 


establish a common approach to coaching for all supervisors, jee yoatehoulla dale. 


such as implementing inquiry cycles. In both 2016 and 2017, 


more than half of the principals surveyed indicated that their Principal (2016) 
supervisors usually or always used a specific coaching 
approach. 


Still, ambiguity and confusion existed in some districts 
Cleveland required supervisors around the meaning of the term coaching and how integrated 
to establish coaching plans for coaching should be in supervisors’ work in their schools. In 
each of their principals and one district, for example, a central office staff member 
align their coaching plans with explained that coaching was not grounded in a common 
UN meaning. Across districts, supervisors described different 
prior year Ss supervisor trainings. 3 F a r . j : 
frequencies and intensities of their coaching efforts in working 
with their principals. Not surprisingly, some principals 
reported regular and intense coaching from their supervisor, but other principals rarely felt that 
they were coached. One principal, whose colleagues in other schools described receiving regular 
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coaching, reported watching a Wallace Foundation video about principal supervisor coaching 
with inquiry cycles and thinking, “Is this what is supposed to be happening? That would be 
great” (2016). 


Some supervisors in each district described differentiating their coaching approach based 
upon their assessment of principals’ needs. For example, one supervisor explained that coaching 
with his principal who was “really, really strong” could focus on more complex tasks, whereas 
coaching with his second-year principal was simply “functional coaching” or “providing support 
for the things that are wiping [the principal] out at this moment” (2016). 


Principals receiving more frequent, more intensive coaching often reported close working 
relationships and familiarity with their supervisors. These principals found that their supervisors 
knew what was going on in the principals’ schools, understood the principals’ goals, and 
perceived their own success as linked to that of their principals. 


D. Principal supervisors led principal networks and professional learning 
communities 


The PSI envisions that as supervisors focus more on instructional leadership, supervisor-led 
meetings with principals will promote principal learning and development rather than simply 
serving as venues to disseminate information. 


Principals in every district nearly uniformly 
nae reported that their supervisors organized PLCs, and 

In some districts, the common use of tools 88 Corncinals in. 2017 d dh ed dh 

served as a means to develop shared perrent 3 poe Sere ees 

vision and routines in areas such as walk- opportunities as somewhat or very useful. Although 


throughs, coaching, and evaluation. For disseminating district information remained a part 
example, sae baiee ae set KN) of these meetings in every district, as had been the 
eee feeae are lick oa basi case before the PSI, most supervisors reportedly 
template provided common language and included professional development and/or job- 
provoked conversations among embedded learning opportunities, which was not 
Siu yol-la Sve) scyeelolelel a ele) igiit-lpr-ls)e)ge)-(epi-suce COMmon before the PSI. Most supervisors across 
coaching and interventions. districts now included elements such as school 


walk-throughs, guided data analysis, and whole- 
group professional development in their PLCs. 


Before the PSI, principal gatherings were oriented 


primarily toward disseminating information, although Gave Before the PSI, 
some districts had begun to focus principal meetings on principal gatherings 
developing principals. Some districts had to significantly were oriented 
change the scope of their principal PLCs when primarily toward 
implementing the PSI. Minneapolis, for example, rarely disseminating 
convened principals in PLCs before the PSI other than to information; now 


network meetings are 
focused on principal 
learning and 
development. 


disseminate information from the central office. Across 
Minneapolis networks, PLCs now included group 
professional development and a targeted walk-through of a 
network school. Des Moines similarly transformed its 


39 


A NEW ROLE EMERGES FOR PRINCIPAL SUPERVISORS VANDERBILT UNIVERSITY / MATHEMATICA POLICY RESEARCH 


information-delivery meetings into principal professional development that included monthly 
school visits in each network, an activity that served as the centerpiece of PLC meetings. 


The targeted nature of PLC activities varied across districts. Most districts gave their 
supervisors significant discretion to organize PLCs. A supervisor in one of these districts described 
designing each session to meet the needs of his principals. In certain cases, districts dictated much 
of the PLC content. For example, one supervisor explained that the curriculum office provided 
slide presentations that all supervisors had to use during PLC meetings. 


E. Principal evaluation systems continued to develop to align with 
supervisors’ changed roles 


Since the start of the PSI, every district had worked to make principal evaluation a process 
that could provide principals with more formative, actionable, and timely feedback. Efforts to 
overhaul principal evaluation, or work within existing evaluation structures, was ongoing. 


Two districts redesigned the official evaluation process. For example, Long Beach 
implemented a new evaluation process in the 2014—2015 school year to facilitate formative 
assessment throughout the year, and later made additional changes to better support alignment 
between the standard evaluation process and the action plans of individual principals. 


Other districts sought to improve the usefulness of existing evaluation systems. Supervisors 
in some such districts worked with district leaders to develop ways to emphasize particular 
aspects of the evaluation to promote formative and actionable feedback. For example, Cleveland 
supervisors explained an effort to create ongoing feedback within the boundaries of the formal 
evaluation process by designating “power standards,” or focus areas from the evaluation rubric 
that they identified as particularly important. Minneapolis supervisors engaged in similar efforts 
to accommodate an evaluation system that “is [not] the best tool, but it’s the tool we’ve got for 
now” (2016). 


Principals’ perceptions of the overall average quality of their evaluation systems in the third 
year of the initiative was 2.8 on a 5.0 scale (Figure V1.6). Principals in most districts suggested 
evaluation was broadly informative but pointed to supervisors’ formative feedback as more 
helpful for improving day-to-day work; principals did not feel the evaluation was a great 
indicator of their ongoing performance or a guide for improvement, with the exception of one 
district. In a second district, a few individual supervisors developed approaches to integrate 
principal evaluation into their work as a learning tool. In general, principal evaluation did not 
typically provide actionable information that supervisors used. 
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Figure VI.6. Principals’ perspectives on the quality of district principal 
evaluation systems, 2017 
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Figure reads: Principals in Des Moines rated the quality of the principal evaluation system an average of 3.1 of 5.0 on 
a five-point Likert scale in 2017. 


Source: Principal surveys, 2017 (N = 611). 


Note: Scale created from four items: “The principal evaluation system is too cumbersome.” “There are too many 
indicators attached to the principal evaluation system to be useful.” “It is unclear how principal evaluation 
data are used in this district.” “The principal evaluation system provides actionable feedback to improve my 
leadership.” See Appendix A.4 for complete details. The overall average was unchanged from 2016. 
District-level averages changed as follows from 2016 to 2017: Des Moines -0.1; Baltimore +0.3; Broward 
+0.1; and Cleveland -0.2. Long Beach and Minneapolis averages did not change from 2016 to 2017. 


Regardless of how principals viewed the evaluation 


system in their districts, they did not perceive O--= Both supervisors and 
supervisors’ dual roles as both evaluator and coach as principals reported 
problematic because of the broader shifts in the that supervisors were 
supervisor role. Specifically, the increased frequency of able to balance their 
contact between supervisors and principals and ongoing dual roles of coach 
feedback helped build trust between them. Across and evaluator. 


districts, principals emphasized that they trusted their 
supervisors to function as both supporters and 
evaluators. As one Cleveland principal explained: 


You don’t feel as though it’s your boss evaluating you. So it’s very comfortable. 
He’ll come in, he’ll have a conversation with you.... He always asks, “How can I 
support you? What do you need from me?” It’s more of that than a formulated, 
check the box. (2016) 


F. Districts and supervisors confronted challenges to changing the role 


PSI districts have made progress in redesigning the day-to-day work of the supervisors to 
support principals, but they have faced a number of challenges in aligning the work of the 
supervisors with the goal of improving instructional leadership and with their revised job 
descriptions. 


Supervisors face tradeoffs in how best to use their time. In the PSI districts, principal 
supervisors were high-level district leaders who must engage in high-level district decisions— 
often decisions that affect instruction and leadership in schools. Yet that work came at the 
expense of time working directly with principals. In one district, for example, principal 
supervisors met biweekly with the chief of schools, attended cabinet meetings with the 
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superintendent, and sat in on some instruction-related departmental meetings or committees. 
Almost all of this work could be justified as indirectly supporting principals because supervisors 
acted as intermediaries for instructional leadership support, but supervisors found it challenging 
to devote time to direct work with principals while also attending to district-level responsibilities. 


Balancing consistency of practice and autonomy is a dilemma. Supervisors in several 
districts had significant leeway to determine how best to work with principals to support their 
individualized needs, but this autonomy also resulted in inconsistency in supervisors’ approaches 
to supporting and developing principals and the quality of that support. In one district, for 
example, principals widely praised the effective coaching during walk-throughs provided by one 
supervisor, whereas principals assigned to another supervisor in the same district reported that 
the supervisor conducted unhelpful walk-throughs with no apparent agenda. Principals in 
districts with a more standardized walk-through approach reported less variation in the quality 
and usefulness of walk-throughs. 


Differentiating supports for principals is difficult. Across districts, effectively 
differentiating the types and amount of supports for principals remained an open question that 
supervisors typically navigated with limited support or guidance. Some supervisors oversaw such 
challenging schools that they spent their time and energy on only a few schools, at the exclusion 
of others in their network. In some cases, supervisors had developed processes for differentiating 
amounts and types of supports necessary for each of their schools, but these practices were not 
widely shared. For example, several Cleveland supervisors relied on coaching plans they 
designed in accordance with the needs of each of their principals and adapted throughout the 
year. All Long Beach supervisors provided a minimum number of visits to all schools and 
conducted additional visits as needed. 


Developing useful principal evaluation systems is ongoing. Principals noted that their 
evaluation systems have room for ongoing improvement. Some principals reported that they do 
not receive ongoing, actionable feedback and are unaware of how their districts use evaluation 
data. Many principals and supervisors felt that the principal evaluation system did not inform the 
supervisor’s work. Supervisors in all but one district tended to describe evaluation as a 
summative measure rather than a formative process for driving improvement. 
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Vil. DEVELOPING SYSTEMS TO IDENTIFY AND TRAIN NEW SUPERVISORS 


A key component of the Principal Supervisor Initiative (PSI) is to develop systems to 
identify and train future principal supervisors to meet the role’s new expectations. Districts that 
have implemented these systems have reported early benefits; a leader in one district described 
its new apprenticeship program as “a signature accomplishment” in the PSI work. 


A. Three districts have developed principal supervisor apprenticeship 
programs 


Three districts have developed fully functioning principal supervisor development or 
apprenticeship programs. In 2016-2017, Cleveland’s Aspiring Network Leaders program, 
Broward’s Director Intern Program, and Long Beach’s Aspiring Directors Program each were in 
their second year of operation. The districts’ approaches to these programs varied, but each 
featured rigorous selection procedures and sustained opportunities for principal supervisor 
candidates to learn the requirements of the role, receive coaching, and participate in different 
aspects of the role through field experiences. However, in none of these districts was completion 
of the program a requirement of becoming a principal supervisor; each of the districts had hired 
principal supervisors who had not completed the program during the first two years of their 
implementation. 


The other three districts had not yet implemented a prospective principal supervisor 
identification and training program, and district leaders noted various reasons for the delay. In 
one district, leaders did not foresee much turnover among their current principal supervisors in 
the next few years. If turnover occurred, they planned to draw new supervisors from the ranks of 
principals currently leading a major district initiative. In a second district, succession planning 
was on hold while a new superintendent was installed. In a third, leaders noted they did not have 
the capacity to focus on this component given other district priorities, such as building a robust 
principal pipeline. In addition, in two of the three districts without a supervisor apprenticeship 
program, leaders suggested that developing new leader tracking systems would help them 
identify principal supervisors and they preferred to build those data systems before engaging in 
developing a program for supervisor apprenticeships. 


In the rest of this chapter, we describe the features of the principal supervisor apprenticeship 
programs in the three districts that have implemented them. 


B. Districts developed various approaches to select candidates for the 
apprenticeship program 


Apprenticeship program participants we interviewed (as current principal supervisors) 
described program selection procedures in two of the three districts as “extensive” and “intense.” 


e In Cleveland, the selection process included a series of interviews, an exercise to 
demonstrate how the applicant thought about and analyzed data, a mock coaching 
conversation, and an essay describing why the applicant wanted to be a principal supervisor 
and identifying his or her strengths and areas for growth. 
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e In Broward, the selection process included an initial résumé review; a writing exercise in 
which applicants responded to a leadership scenario; a role-play exercise in which 
applicants watched a video of a difficult school situation and discussed how they would 
coach a principal through it; and a multiphase interview with current supervisors, the 
director of leadership development, and the director of service quality. 


e In Long Beach, the selection process consisted of a review of résumés and letters of intent 
along with interviews. The process emphasized principal evaluation ratings, and prioritized 
high ratings in areas related directly to teaching and learning, which one central office leader 
described in 2016 as “the foundation to their work.” 


Current principal supervisors served an important role in selecting program participants. In 
addition to participating in screening and interviews, they played an informal role in encouraging 
or discouraging principal applications and helping potential candidates think about areas for 
development with the goal of future work as principal supervisors. 


C. Apprenticeship programs offered multiple types of training opportunities 


Although the three districts implementing yearlong apprenticeship programs used different 
models, all districts kept the program cohorts small. In Long Beach, four aspiring supervisors 
participated in the program in the first year and three in the second year; in Cleveland, three 
completed the program in the first year and two in the second year; and in Broward, six 
principals participated in the program each year. 


Across programs, other common hallmarks included opportunities to experience actual 
aspects of the work of the principal supervisor, mentoring by current supervisors, and coaching 
and feedback for growth and development. 


Cleveland designed its program to address gaps in principals’ transitions into the principal 
supervisor role that the district had observed previously, such as challenges with coaching and 
building strong supervisory relationships. The training program consisted of three components. 
First, apprenticeship program participants attended monthly training meetings that included a 
book study, conversations about coaching, and problem-solving exercises around scenarios 
related to instructional leadership. Second, participants engaged in a coaching internship to coach 
another principal throughout the year. In the second year, the program was revised and 
participants coached multiple principals rather than a single principal to better mimic the 
principal supervisor role. All of the principals selected for coaching were on improvement plans 
or otherwise formally required to participate in coaching. The program provided structure for 
coaching by establishing a six-week cycle for aspiring supervisors to set specific goals and 
support the principals in particular areas. Third, a current supervisor mentored each aspiring 
supervisor. Participants described some variation in the robustness of these mentoring 
relationships. The best-developed mentoring relationships included school visits to observe 
supervisory activities, experience designing professional development for principals in 
collaboration with the mentor, and opportunities for feedback. 


Broward’s apprenticeship program emphasized opportunities for hands-on learning. The 
training program included four components. First, interns participated in trainings on the 
district’s inquiry cycles and coaching processes. Second, interns worked alongside a principal 
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supervisor for the year with a focus on intensively supporting six schools in the supervisor’s 
portfolio. This component provided participants experience in the role and, in particular, helped 
to develop their coaching skills. One participant described the process as a “gradual release” in 
which the intern eventually took the lead on providing school support. The supervisor acted as a 
mentor, sometimes joining the intern on school visits, providing feedback, and preparing the 
intern for the practical aspects of the supervisor role through conversation and opportunities to 
shadow. Third, interns received one-on-one coaching from the Center for Educational 
Leadership, which included providing feedback during visits to the schools the interns supported. 
Finally, participants joined other principal supervisors in their central office liaising duties on a 
rotating basis to learn about the central office work, such as participating on committees, leading 
initiatives, and interacting with other departments. 


Long Beach’s program had some similar elements to those of Cleveland and Broward. 
Participants engaged in monthly training meetings. They also closely observed principal 
supervisor work by accompanying current supervisors on school site visits (that is, job 
shadowing), attending supervisors’ lab days—which are intensive learning opportunities for the 
supervisors—and joining collaborative inquiry visits to schools. In addition, participants attended 
monthly professional development meetings and were responsible for co-planning and 
implementing support meetings for early-career principals. As one participant noted, this “starts 
to get you out in front of your peers in that support role” (2016).The program provided multiple 
opportunities for receiving feedback from various district leaders, including one-on-one meetings 
with the deputy superintendent of schools. Long Beach developed its apprenticeship program 
with the aim of providing an overview of the district and developing an understanding of 
principal supervisor work from the perspective of “systemness,” rather than from the perspective 
of a principal in one building. 


D. Districts identified a number of benefits of apprenticeship programs 


District leaders and participants identified a number of 


i : oe : . : Q To become a [principal 
additional benefits of the principal supervisor apprenticeship [princip 


supervisor], they still have 


programs. First, although leaders in all three districts with to apply and interview. This 
apprenticeship programs noted that completing the program just gets them into the 

was not a guarantee of a principal supervisor position, it was program and exposed to 
helpful for selection. District leaders gained much more the professional 
information about the participants to use in selection development ... when they 


go through the interview it 
is exactly the same as 
anybody else. 


decisions than would typically be available during the hiring 
process. In some cases, district leaders noted that the 
program helped identify weaknesses that disqualified some ne See 
candidates or directed further development for candidates District Leader (2016) 
selected as supervisors. 


Moreover, district leaders described program completers as more prepared than new 
supervisors had been before the PSI. The program helped participants address skill deficiencies 
in areas such as coaching that previously had been common among many new supervisors. The 
programs also facilitated succession planning, including estimating the number of new 
supervisor openings in coming years and specifying steps necessary to ensure a pipeline of 
effective candidates for those openings. 
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Similarly, for participants, the program enabled them to hit the ground running, with a much 
clearer picture of the expectations and challenges of the supervisor role. Participants also noted 
that they forged valuable new connections to other leaders in the district, learned more about 
district operations and where to seek resources for their schools, and honed skills relevant to 
principal support. 


Long Beach staff noted that its apprenticeship program deepened participants’ knowledge of 
the central office and how it worked, which was important even if the participant was not 
immediately placed in a principal supervisor or other central office position. Similarly, the 
program helped central office staff stay connected to what was happening in schools through the 
feedback loops resulting from their interactions with program participants. 


E. Districts faced some challenges in building a robust principal supervisor 
apprenticeship program 


Some PSI districts have made progress in developing principal supervisor apprenticeship 
programs, but these districts have also faced a number of challenges implementing these 
programs. 


Districts do not always have open supervisor positions. A common challenge 
encountered by the three districts was a surplus of program completers relative to the number of 
principal supervisor positions available. A district might invest in building skills in potential 
supervisors that might not be maintained if not used. District leaders noted that this problem 
compounds as more people complete the program, particularly after a district achieves its goal of 
having a stable set of principal supervisors. A leader in Long Beach explained in a 2016 
interview that this concern is one reason the district kept the apprenticeship program small: “You 
have to think about how deep you want this bench based on the number of positions that may 
potentially be available, and you don’t want people sitting in the pool forever.” Cleveland’s 
approach provided one possible solution to this challenge: the district provided two program 
completers with special projects to lead while continuing as principals, which also enabled the 
program completers to develop in new areas. 


Creating time for program activities is challenging. Time constraints for apprenticeship 
participants were another common challenge. In both Cleveland and Long Beach, participants 
completed apprenticeship work while simultaneously serving as principals. Participating in the 
program required those principals to spend time out of their buildings and complete many tasks 
after school hours in addition to their principal duties. As one supervisor who participated in 
their district’s apprenticeship program noted: 


The workload was crazy because [I had to] miss that many half days. It puts a lot 
of pressure on your site, puts a lot of pressure on your assistant principal... 
[I] was kind of basically working two jobs. (2016) 
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Broward’s program model demonstrates a possible 


path forward to address this challenge. The district Gr Districts can use 
moved the principal participating in the program to full- cr eative Staffing 
time work in apprenticeship training activities and filled Strategies to free up 


time and resources 
for aspiring principals 
to participate in 
apprenticeship 
programs. 


the principal’s spot with an assistant principal from the 
district’s aspiring principals program. The district then 
placed an aspiring assistant principal (a teacher) in the 
vacant assistant principal position. After these shifts, 
only a teacher opening remained. As such, the district 
equated the cost of the training for the principal 
supervisor with the cost of a first-year teacher’s salary. 


Quality varies within the same district for participants. By design, apprenticeship 
programs were small, and part of their success rested with close mentoring relationships between 
participants and current supervisors. Some participants, however, raised concerns about the 
consistency of mentoring and coaching afforded to them. 


Districts need to encourage strong applicants. Districts grappled with how to encourage 
the most promising candidates to apply for the apprenticeship program. In one of the districts, 
participants felt the opportunity to apply was communicated well and broadly, but in the other 
two, participants and district leaders expressed concern that the apprenticeship program was not 
yet well known and that identifying potential participants was not sufficiently systematic. As one 
central office leader noted: 


We need to think about asking people and not just letting it be ‘apply if you want 
to’—if we see someone who is an up-and-comer or who would be a great central 
office leader, we need to be more proactive in getting them to the program. (2016) 


Programs take good principals out of schools. Some district leaders wrestled with the 
balance between a desire to train the best candidates as supervisors and concerns about whether 
schools can spare the loss of the most effective principals to supervisor positions. This concern 
was particularly acute in districts lacking a robust apprenticeship for school-level leaders, which 
made finding a strong replacement principal more difficult. 
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Vill. STRENGTHENING CENTRAL OFFICE STRUCTURES 


Toward the end of the initiative, the Principal 
Supervisor Initiative (PSI) required districts to 
create a detailed plan for reorganizing central 
offices so that districts could “independently 
support the systems put in place” to sustain changes 


Other central office roles 


Supervisors in some districts also held 
other formal roles within the district office 
beyond their work overseeing principals. 


after the initiative ended. Yet very early in the Almost half (49 percent) of supervisors 
initiative, districts discovered that more immediate recalled having at least one additional 
changes to the work of the central office would be formal role in 2015. This proportion fell to 


necessary to facilitate the change to the supervisor ele Cem UNG Ect alee ei 'crele Niele) 
formal roles included department liaison, 


role and set about implementing oe of those new principal induction facilitator, and 
changes. Here we describe the reorganization of summer school administrator. 
central office structures to facilitate the supervisors’ 
work and new vision for the role. 


A. Overview of central office structures 


Principal supervisors worked in unique and different organizational structures across the six 
PSI districts (Table VIII.1). At the time of data collection, all six PSI districts housed supervisors 
within a single central office department, part of an office of principal and school supervision. 
Typically, separate departments oversaw teacher development and curriculum. Most supervisors 
reported to a cabinet-level chief officer who was responsible for overseeing and evaluating their 
progress as well as providing formative feedback throughout the year. 


Table VIII.1. Central office organizational structures for supervisors in the PSI 
districts, 2016-2017 school year 


Levels removed 


Immediate from 
Supervisor title Department supervisor superintendent 
Baltimore Instructional leadership Office of Academics Chief academic Two? 
executive director (ILED) officer 
or senior ILED 
Broward Cadre director Office of School Chief of OSPA Two 


Performance and 
Accountability (OSPA) 


Cleveland Network support leader Office of Academics Chief academic Two 
officer 

Des Moines Director Office of Schools Executive directors Three 
(two) 

Long Beach Assistant superintendent Office of Schools Deputy Two 

or director superintendent of 

schools 

Minneapolis Associate superintendent Office of Schools Chief of schools Two 


Sources: District interviews; artifacts collected during site visits (Such as job descriptions and district organizational 
charts). 


lnitially, the supervisor position in the district was three levels removed from the district superintendent. 
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Supervisors’ staff and direct reports. Most supervisors worked with at least one assistant, 
and the average number of assistants reported—three—remained the same over the first three years 
of the PSI. Assistants included secretaries, instructional facilitators, and other administrative staff 
who did not report directly to the supervisors, but assisted them in their work. 


About half of supervisors reported having at least one other direct report in addition to 
principals. For example, supervisors in Cleveland hired their own “Barrier Breaker” and “Action 
Team Coach,” both of whom reported to that supervisor and worked exclusively with the 
supervisor’s principal network. According to supervisor surveys, the average number of direct 
reports to supervisors (excluding their principals) declined from 2016 to 2017, from five to three. 
Fewer direct reports are consistent with the goals of the PSI, to enable supervisors to focus their 
time and attention working with principals rather than supervising support personnel. 


B. Central offices reorganized to support principal supervisors 


ee the first aa se PSI, districts enacted a Districts made early 
variety of structural and cultural changes meant to improve changes to the 


and facilitate central office support for supervisors’ work central office to 


with principals (Table VIII.2). The specific changes support the changes 
depended upon the unique context of each district and the to the principal 
existing structure and capacity of the central office. supervisor role. 


Districts reallocated nonmanagerial responsibilities to help supervisors focus on 
instructional leadership. All six districts created new roles or adjusted the roles of other central 
personnel to absorb noninstructional duties formerly handled by principal supervisors. 


Three districts created new positions to assume operational responsibilities: 


e Minneapolis created a deputy superintendent of operations position in 2015 to take on many 
of the operational issues that formerly went through principal supervisors. 


e Broward created an Office of Service Quality to handle parental complaints and school 
improvement processes such as accreditation and state compliance reporting. 


e Baltimore hired a building manager to take over school operational and maintenance issues. 


The remaining three districts chose to rely on existing personnel to absorb the 
noninstructional responsibilities previously assigned to supervisors by reinforcing or altering 
their job descriptions: 


e Des Moines changed the role of its executive directors, who supervised principal 
supervisors, to include the operational and logistical responsibilities that had previously 
fallen under the purview of the principal supervisors. 


e Cleveland supervisors oversaw barrier breakers, a position assisting supervisors by handling 
some operational and logistical needs for schools directly. Barrier breakers became 
designated contacts for principals seeking miscellaneous logistical help. 


e Long Beach transitioned three hybrid principal supervisors to central office-only positions. 
All three individuals continued to work closely to support and develop principal supervisors, 
but they no longer supervised principals. 
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Table VIII.2. Districts’ major central office changes before and during the first three years of the PSI 


District 


Before PSI grant 


2014-2015 school year 


2015-2016 and 2016-202017 school years 


Baltimore Created two senior principal Created central call center for principals to Created departmental liaisons to assist 
supervisor positions to lead and call directly with their problems rather than principals directly 
support regular principal supervisors go through principal supervisors Consolidated senior principal supervisor role 
into a single position 
Broward Created Office of Service Quality to Developed an online platform that Moved Leadership Development department 
handle operational issues delivered weekly communication to from Talent Development department to 
Created departmental liaison roles to principals to streamline communication supervisors’ department, Office of School 
promote exchange of information Performance and Accountability 
between supervisors and other Split supports throughout central office 
departments departments in elementary and secondary 
levels, with one person atop each structure 
Gave principals increased opportunities for 
feedback 
Cleveland Established barrier breaker position to Implemented network support teams to Continued strengthening network support teams 
provide school logistical support assist supervisors and principals directly 
Established action team coach to 
provide teacher instructional support 
Des Moines None reported Replaced key personnel in some Established network support team for each 
departments principal supervisor 
Replaced personnel in Office of Academics 
Redefined role of executive directors 
Dispatched weekly information newsletter to all 
district personnel and principals 
Long Beach None reported Moved some operational tasks to Removed hybrid central office and principal 
administrative assistants supervisor roles — now central office only 
Implemented school-based budgeting Split secondary cohorts into middle school and 
high school cohorts and added additional 
assistant superintendent 
Minneapolis Hired chief of schools to oversee Created deputy chief of schools position to Hired permanent superintendent 
supervisors handle operational matters Adopted new school budgeting formula 
Added parental complaint ombudsman 
position 
Note: Data for changes before the grant and changes made in the 2014—2015 school year were collected via interviews in summer 2015. Data for the 2015-2016 and 


2016-2017 school years were collected via interviews in fall 2016. 
PSI = Principal Supervisor Initiative. 
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Principal supervisors reported that steps taken to none ; 
reorganize the central office to absorb noninstructional Gr District efforts to shift 
responsibilities helped their work. In 2017, a slightly non-instructional 

: ; responsibilities to 
higher percentage of supervisors (40 percent) agreed that 

heduli f offi . bled th other central office 
scheduling of central office meetings enabled them to _ staff freed supervisors 
spend more time in schools, compared with 33 percent in to focus on principals’ 
2016 (Figure VIII.1). Supervisors reported less intrusion instructional 
on their instructional work with principals. The leadership. 
percentage of supervisors who agreed that the 
organization of the central office interfered with their 
ability to work with principals also fell, from 51 percent in 2016 to 36 percent in 2017. In 
addition, principal supervisors were much less likely to report that central office organization 
interfered with their ability to work with their fellow supervisors (44 to 25 percent agree or 
strongly agree; not shown in figure). 


Despite the efforts by districts to shift responsibilities from supervisors to other personnel, 
supervisors reported that their principals were not always sure who to contact when they needed 
assistance. Although the percentage of principal supervisors who agreed that principals sought 
help from them because they did not know who to contact in the central office fell 7 percentage 
points from 2016 to 2017, 46 percent of supervisors in 2017 still agreed or strongly agreed that 
this was the case. Some supervisors continued to report that central office demands for their time 
and participation in non-essential meetings reduced the amount of time they spent in schools. 
Indeed, supervisors universally reported that managerial meetings were too frequent. 


Figure VIII.1. Principal supervisors’ perceptions of central office organization, 
2016 and 2017 


Central office meetings were 2016 
scheduled so | could 
maximize my time in schools. 2017 


The way the central office is 2016 
organized interferes with my 
ability to work with principals. 2017 


Principals seek help from me 2016 
because they do not know who 
to contact in the central office. 2017 


0 10 20 30 40 50 60 
Percentage of supervisors 


mAgree mStrongly agree 


Figure reads: 29 percent of supervisors agreed and 4 percent strongly agreed that central office meetings were 
scheduled so they could maximize their time in schools in 2016. 


Source: Supervisor surveys, 2016 (N = 49) and 2017 (N = 50). 
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District reorganization focused on fostering collaboration and coordination across 
departments. Districts recognized early on that the success of the PSI would depend on efficient 
coordination between principal supervisors and other departments within the central office. 
Increased collaboration and coordination across 
departments can provide more unified and responsive Many districts 
services to schools and can help supervisors stay apprised Gr created central office 
of the work of other central office departments to help structures designed 
them support schools. Districts developed two approaches to improve — 
to increase central office collaboration and coordination: a eee oii 
support teams and liaison roles. Both structures promoted Domne ee 
cross-departmental communication and the integration of departments, ond 
central office departments with the work of schools and schools. 
supervisors. 


Support teams. In a support team structure, supervisors were assigned dedicated 
representatives from other departments who meet with them regularly to discuss ongoing work, 
attend their principal network meetings, and work directly with their principals (Figure VIII.2). 
Support teams made services from central office departments more readily accessible to 
principals and supervisors. 


Figure VIII.2. District support team structure 


Human Special Curriculum/ 
Resources Education Instruction 


Support team made up of representatives of central office departments 


Supervisor 


Source: District interviews with central office personnel and principal supervisors, 2015 and 2016. 


Two districts, Cleveland and Des Moines, implemented support teams for all supervisors; 
Minneapolis created a support team for the supervisor overseeing turnaround schools. The 
typical support team in Cleveland included representatives from the departments of Human 
Resources, Budget, and Instructional Technology, among others. These representatives joined 
supervisors during their principal network meetings and occasionally accompanied them on 
walk-through visits to school buildings. 
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Principals were encouraged to contact their support team representatives directly rather than 
through their supervisors, an arrangement that had the added benefit of keeping supervisors’ time 
focused on work with principals on instructional matters. Support team members became 
familiar with the schools and their needs via consistent group interactions. A principal with 
facilities problems could call on his or her representative at the facilities department directly for 
help rather than contacting the supervisor. Although principals reported occasional 
communication breakdowns that required them to loop in their supervisors, they perceived the 
system of network support teams as functional and efficient. 


Principal supervisor liaison roles. Under a principal supervisor liaison support structure, 
each supervisor was assigned to be a liaison to another department (Figure VIII.3). The 
supervisor attended meetings and served as an information bridge for that department. 


Figure VIII.3. District liaison role structure 
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Source: District interviews with central office personnel and principal supervisors, 2015 and 2016. 
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Each principal supervisor in Baltimore and Broward and a subset of supervisors in Long 
Beach was assigned to serve as a liaison to central office departments. Supervisors attended 
meetings and communicated with personnel from the department to which they served as a 
liaison. The liaison structure aimed to improve coordination among departments and reduce the 
tendency of departments to work independently of one another. Communication was two-way: 
principal supervisors represented the interests of their principals and home department, but also 
carried an understanding of their partner department’s agenda back to other supervisors in their 
home department to share with other supervisors. 
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The central office worked to improve overall systems 


of central office communication. PSI districts changed 
central office processes to increase the quality of 
communication among central office departments, 
supervisors, and schools: 


Des Moines created a streamlined system for 
communication, a Monday Memo from all departments 
in the central office to all district personnel, including 
principal supervisors and schools. Des Moines’ 
personnel were overwhelmingly positive about this new 
system and felt that communication to schools had 
improved dramatically. 


Broward streamlined communication by bundling 
messages across departments so principals would not 


Q We used to operate in very 


much a siloed organization, 
and you knew that person 
was in special education, 
but you really didn’t know 
what they did and how they 
were contributing.... Now 
we have a much better 
understanding from role to 
role and department to 
department around what 
that person does and what 
their function is, and how 
we can better support and 
leverage their resources in 
schools and for principal 
supervisors. 


receive conflicting information from different sources. 
The head of the supervisors’ department, the Office of 
School Performance and Accountability, personally 
screened communications from several departments and posted them to a web system for 
principals to view all together. 


Principal Supervisor (2016) 


e Cleveland often invited network support team representatives to present at principal network 
meetings in addition to joining supervisors and principals for building learning walks. 


e Long Beach principals and supervisors began attending the same district meetings and 
seminars, and therefore received consistent messages from district leaders. 


C. Districts faced challenges to strengthening the central office 


Districts confronted a number of challenges as they worked on strengthening the central 
office to support principal supervisors and their new roles. 


Central offices strived to develop school-centered orientations. Despite progress in 
improving key areas of central office function, officials in all districts acknowledged the 
difficulty of creating an overall school-centered orientation across the central office. 


Gu Supervisors’ 

perceptions of the 
central office support 
for their role became 


more positive 
over time. 


A large majority of principals in 2016 and 2017 
believed improving teaching and learning was a key 
focus of the central office (Figure VIII.4). However, 
some principals did not consistently view the central 
office as oriented toward supporting schools. Fewer than 
half (44 percent) of principals agreed or strongly agreed 
that the central office was organized to support principals 
in 2017, compared with 35 percent in 2016. In addition, 
most principals each year agreed or strongly agreed that 
requests from the central office took time from their 
focus on teaching and learning. 
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Figure VIII.4. Principals’ perceptions of central office orientation, 2016 and 2017 


Challenge: Principals reported slow change in orienting 
central office to support their work 


Improving teaching and learning isa 2016 
key focus of the central office 2017 


learning because of requests from 
the central office 2017 


The central office is organizedto 2016 
support me in my roleasa principal 5947 


| lose time focusing onteachingand 2016 [3 4x2 
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Figure reads: 40 percent of principals agreed and 17 percent strongly agreed that improving teaching and learning 


was a key focus of the central office in 2016. 
Source: Principal surveys, 2016 (N = 630) and 2017 (N = 611). 


Across districts, principals and principal supervisors viewed compliance-oriented 
departments such as Human Resources, Legal, and Special Education as more out-of-touch with 
the work of schools and the new work of principal supervisors and less likely to “get on board,” 
as one principal noted, with a school support mission. 


Districts faced variation in central office departments’ performance and 
responsiveness. Principals, principal supervisors, and central office officials in all districts 
acknowledged that certain departments were ill equipped or less willing than others to support 
the changes and perspectives of the PSI and supervisors’ revised roles. In interviews, personnel 
at all levels reported that variation in departmental performance prevented the central office, 
including principal supervisors, from taking a completely unified approach to school support, 
undercutting the potential of the new structures. These personnel often expressed frustration at 
the lack of accountability for some departments. As one district leader noted: “Something that 
was very troubling to me as a building principal was there was apparently no accountability 
moving upstream. If you didn’t get something you needed, too bad” (2015). Respondents 
sometimes attributed poor departmental performance and lack of responsiveness to staff 
turnover. More than one-third of principal and supervisor survey respondents in 2017 indicated 
that turnover affected the ability of the central office to support principals. Although respondents 
typically viewed intentional personnel changes in positive terms, they considered unplanned 
turnover a threat to implementing the PSI work successfully, particularly for collaborative 
structures such as support teams and liaison roles. 


56 


A NEW ROLE EMERGES FOR PRINCIPAL SUPERVISORS VANDERBILT UNIVERSITY / MATHEMATICA POLICY RESEARCH 


Supervisors encountered political tensions. Political tensions often complicated and 
impeded the ability of supervisors to support schools. Supervisors typically used the term politics 
to refer to the informal brokering of power within districts. Supervisors worked to shield 
principals from political issues that would detract from their leadership. As one supervisor 
explained, “I’ve always said to principals, ‘Don’t let the political stuff get in your way of doing 
what you need to do” (2016). 


Likewise, political instability in the central office bred a lack of ownership and tendency to 
circumvent sensitive issues to save face, at the expense of school improvement. To conduct their 
work, supervisors often had to resort to informal power-brokering with individuals in other 
offices with whom they had developed relationships. New supervisors in these districts often 
remarked on their relative disadvantage, as a lack of familiarity with other individuals in the 
central office prevented them from effectively supporting their principals. 
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IX. CONCLUSIONS 


The Wallace Foundation’s Principal Supervisor Initiative (PSI) challenged districts to 
redefine the principal supervisor position from one that traditionally focused on administration, 
operations, and compliance to a position dedicated to developing and supporting principals for 
school improvement. Before the PSI, few districts invested systematically in principal 
supervisors as a point of leverage for principal support and development. Three years into the 
four-year initiative, PSI districts have substantively changed the supervisor role, changes that 
have implications beyond the supervisor—principal relationship. 


G 


Districts made substantial changes across all five components 


The initiative achieved several noteworthy accomplishments: 


ov 


oe 


Job description. Districts worked with stakeholders to revise the supervisor 
job description to delineate the expectations for the supervisor role to focus on 
supporting and developing principals in schools. 


Span of control. Before the PSI, supervisors oversaw an average of 16 
principals, though this number was as high as 20 in one of the districts. 
Districts reduced supervisors’ span of control within the first three years of the 
initiative to an average of 12, typically by hiring more supervisors. The number 
of supervisors who reported that they oversaw too many principals declined in 
every district. Supervisors were assigned strategically to principal networks to 
provide targeted supports. 


Supervisor training. Districts implemented dedicated and unique training 
programs to develop supervisors’ capacity in coaching and principal support 
and development. The content of these trainings focused on high quality 
instruction and developing instructional leadership, and relied on job- 
embedded approaches to observe supervisors in practice in schools and 
provide feedback. 


Succession planning. Apprenticeship programs, fully developed and 
implemented in three districts, were a key strategy for preparing school leaders 
for principal supervisor positions. These programs featured rigorous selection 
procedures and offered a mix of training sessions, individual coaching and 
performance feedback, mentoring from a current principal supervisor, and 
shadowing of central office leaders. 


Central office restructuring. Districts made substantial progress in 
implementing approaches to central office change in the initial years of PSI. 
Redefining the role of the principal supervisor required changes in central 
offices as supervisors’ noninstructional responsibilities shifted to others. 
Districts used the opportunity to streamline central office departments, 
dismantle barriers that stifled communication between departments, and 
improve processes for principals to access supports and resources. Districts 
improved at making central office departments aware of the work of 
supervisors. Many of these changes, especially the support team structures 
and liaison roles, resulted in better integration and collaboration across 
departments in their efforts to support schools. 
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A. Districts changed the principal supervisor role and enhanced school 
support more broadly 


The daily work of principal supervisors has changed. By the third year of the PSI, 
supervisors spent most of their time in schools, working with principals and other leaders. 
Importantly, supervisors used their time in schools in qualitatively different ways; across 
districts, supervisors reportedly improved the focus and quality of walk-throughs, feedback, and 
coaching offered to principals. Although supervisor practices remain inconsistent in some 
districts, substantive changes in terms of the ways that supervisors enacted the role were evident 
in every PSI district. 


Principals reported more productive relationships with their supervisors. As principals 
and supervisors became more accustomed to the work of the new role over time, principals 
reported deepened trust, improved communication, and improved overall professional 
relationships with their supervisors. Principals often attributed these relational changes to more 
frequent coaching conversations and dialogue with their supervisors as well as the supervisors’ 
increased focus on understanding the specific context of each school. Many participants noted 
that these strong relationships made it possible for supervisors to serve as both coaches and 
evaluators without tension. Moreover, supervisors reported continued efforts to improve the 
usefulness of principal evaluation systems in their ongoing work with principals. 


The initiative was a strategy for enhancing school support more broadly. The changes 
to the principal supervisor role spurred districts to implement structures to enhance school 
support, as district leaders grappled with existing structures that constrained principals’ and 
supervisors’ capacity. Changing expectations via the revised job description created a domino 
effect that spurred changes in other central office roles and systems. 


B. Districts faced ongoing challenges 


Despite their many successes as they developed the new role for their principal supervisors, 
districts also faced some challenges. 


Clarifying and reinforcing a focus for the principal supervisor role. In most districts, the 
sense of what constituted principal instructional leadership was not well defined, despite the 
presence of principal evaluation systems. In some cases, instruction and instructional leadership 
were not Clearly differentiated. Supervisors often varied in how they approached and monitored 
the development of instructional leadership with their principals or did not explicitly focus on 
developing the principal’s own leadership. To address these issues, districts will have to continue 
to clarify, adopt, and specify standards for instructional leadership. They will also have to 
implement supervisor training that aligns with a clear definition of instructional leadership so 
that supervisors can support and develop principal leadership. Tools, routines, and clear 
definitions that focus on the core of the interactions between supervisors and principals can offer 
effective steps to establish non-negotiable aspects of the role. 


Investing other central office departments in the work. The importance of strengthening 
central office roles and structures is paramount. Although many of the changes to the supervisor 
role could be made without consulting or involving other central office departments, supervisors’ 
work continued to depend on the performance of other departments. As districts urged 
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collaborating with and involving other departments in work that had formerly been solely the 
domain of supervisors, some supervisors and principals experienced frustration as some 
departments seemed to lack the capacity to fulfill their new roles. Moreover, supervisors 
perceived a need for continued work to educate personnel in other departments about the new 
principal supervisor role. As of 2017, only 24 percent of supervisors reported on the survey that 
they agreed or strongly agreed that “departments in the central office understand my work” (up 
from 18 percent in 2016). 


Creating change within the unique context of each district. The districts involved in the 
PSI are complex institutions, each of which has evolved over many years to meet the needs of its 
community with the resources it has at hand. District leaders and supervisors often expressed 
sincere investment in efforts to revise the principal supervisor role, but many also lamented the 
challenge of making changes viewed as antithetical to the district climate and culture. Much of 
the work related to changing supervisor role led participants to defy district traditions and 
established ways of working that, at times, created friction. Such challenges underscore both the 
importance for districts of clearly describing how revising the principal supervisor role fits into 
the overall mission of the district and adapting the revisions to suit their individual contexts. 


C. Looking ahead: Areas for continued focus 


Districts will continue to address the challenges of changing the day-to-day practices of 
principal supervisors and build on initial successes. 


Balancing expectations for supervisors with what they can feasibly accomplish. Despite 
efforts to protect supervisors’ time, many continued to feel overburdened. Districts will have to 
consider travel time among schools, availability of support staff, and central office demands for 
supervisors’ participation in meetings or activities as they continue to refine the supervisor role. 
Districts wrestled to identify a balance between supervisors’ central office involvement and time 
spent in schools. Too much time in the central office meant supervisors could not visit their 
schools and provide the intended support for principals. Conversely, too little time in the central 
office left supervisors out of the loop and disconnected from central office departments and 
personnel; these connections ensure their principals are supported. In addition, districts will 
continue to determine non-negotiable aspects of the job, such as the minimum number of visits to 
schools, which could then vary beyond the minimum, according to supervisors’ discretion. 


Articulating expectations for differentiation. Principals will continue to have different 
needs that require supervisors to offer differentiated types and degrees of support. Differentiation 
of practice, beyond time use, remains an important aspect of supervision going forward. 


Striving for consistent quality of supervisor practices. Districts will need to cultivate 
systems to support consistency in quality of supervisor practices. In some districts, principals’ 
experiences with their supervisors varied widely. Efforts such as Long Beach’s collaborative 
inquiry visits make supervisor practice public and support analysis of key supervisor practices 
such as walk-throughs and coaching. Providing supervisors time to develop common tools, 
routines, and definitions for their work can also support consistency in quality practice. 


Developing high quality job-embedded training approaches. Sustainability requires that 
both veteran and new supervisors receive ongoing training. As districts hire new supervisors, 
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they have to develop systems for training new personnel. District leaders will have to continue to 
focus on developing internal capacity to provide high quality, job-embedded training and support 
for supervisors. They will have to determine district roles and responsibilities for providing 
training and support as the resources available for technical assistance providers are reduced. 


Addressing succession planning and designing apprenticeship programs. Districts with 
new apprenticeship programs must address important questions about these programs, such as 
how to make time for participants to engage in them at a high level and how to provide 
opportunities to leaders who complete the program but who are not placed immediately into a 
supervisor role. Districts without apprenticeship programs will need to develop their own 
approaches to succession planning for the principal supervisor role, drawing on the lessons 
learned by the other districts. 


Maintaining momentum to promote sustainability. Maintaining existing momentum and 
focus on the changes underway is important for sustainability and continued growth. Districts do 
not improve over time in a linear fashion; nor should this be expected in a complex change 
initiative. As in any change initiative, districts will have to work to prevent backsliding to old 
routines with sustained effort, continued refinements, and continued championing of the new 
principal supervisor role. 


The next report for the study will examine how districts addressed the ongoing challenges 
from the first years of the initiative, and the changes they made in the final year of the initiative. 
It will also examine how districts’ changes to the principal supervisor role affected principals’ 
performance. 
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Figure A.1. The revised job description, Long Beach, 2015 


TITLE OF TITLE OF NEXT 
JOB TITLE IMMEDIATE MANAGER HIGHER MANAGER 


Instructional Director Assistant Superintendent or Superintendent 
(Principal Supervisor) Superintendent 


Position Summary 


The supervision, support and coaching of principals is the primary responsibility of an Instructional Director. 
This designation means that instructional directors aim to spend up to 80% of their time in school buildings, 
engaging in professional learning with principals or working with district staff to build support systems in 
service to principals. The work of the Instructional Director will be coaching based, whereby principals are 
coached toward effectiveness through clear expectation setting, quality communication and feedback based 
on a foundation of trust. 


Professional Background 


Instructional Directors are leaders who have served as effective principals according to the LBUSD 
Principal Domains and Dimensions and who have demonstrated a rating of “Distinguished” in the 
Teaching & Learning domain. 


Instructional Directors have demonstrated success in raising student achievement. 


Instructional Directors have effectively supported and held staff accountable for meeting LBUSD 
expectations. 


Instructional Directors have demonstrated effective relationships with other principals and district staff 
members through their work as principal and/or district leader. 


Instructional Directors have demonstrated a deep understanding of the urban school system environment 
and have a proven commitment to improving student achievement and district systems to serve all 
students. 


Key Responsibilities 


Collaborate with all principal supervisors to ensure coherence and consistency in the implementation of 
the LBUSD principal evaluation system. 

Work one on one with principals as partners to grow their instructional leadership capacity and to ensure 
that they have the support to attain effectiveness on the LBUSD Principal Domains and Dimensions. 
Provide and broker high quality, research-based professional development for principals based on 
individual and group learning needs. 

Collaborate with other divisions and leaders in the central office to provide and broker necessary 
resources and support to ensure that principals attain effectiveness on the LBUSD Principal Domains and 
Dimensions. 


Act as a role model for principals in all leadership responsibilities. 
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JOB DUTIES 


In order to support all principals in reaching effectiveness on the seven LBUSD Principal Domains and 
Dimensions, instructional directors (principal supervisors) will: 


Teaching and Learning 


Demonstrate understanding of effective instructional practices, intervention strategies and use of 
assessment practices through contributions to district planning and interaction with individual and 
groups of principals. 

Develop and provide professional learning experiences for principals that result in principal use of new 
systems and/or strategies at the school site to enhance teaching and learning among classrooms. 


Reinforce teaching and learning as a primary lever of positive impact on student achievement by 
frequently visiting classrooms with principals and participating in problem solving that helps principals to 
enact high quality professional learning at their school sites. 


Coach principals to build teaching and learning systems at their school site that can be illustrated through 
observable actions, artifacts, documents and/or conversations with staff and students. 


Environment and Equity 


Engage in complex work with principals that promotes high expectations, appreciation and continuous 
improvement for the principal, students and staff. 


Coach principals to identify students at the school site who are struggling, are persistently lower achieving 
or underrepresented and develop action plans to address students’ needs. 


Support principals in developing school wide systems that ensure ALL students advance toward mastery 
of the Common Core State Standards and attainment of college and career readiness. 


Communication and Engagement 


Model effective communication and engagement practices through high quality written and oral 
communication with principals and district staff including well-written supervisory documents, office 
communication and daily correspondence. 


Work as leaders of change in support of district initiatives, school innovation and the pursuit of positive 
student outcomes. 


Engage with internal and external stakeholders to promote schools’ efforts to bring all students toward 
mastery of the Common Core State Standards through a Linked Learning approach and to exemplify the 
LBUSD Graduate Profile upon graduation from high school. 


Supervision, Evaluation, and Employee Development 


Seek opportunities to collaborate and share leadership with principals, including encouraging principals to 
engage in principal meeting presentations, district or conference presentations and/or leadership 
opportunities specific to an individual principal. 


Act as coach and supervisor to principals, upholding the intent of LBUSD evaluation process to ensure 
continuous improvement among principals. 


Demonstrate a working knowledge of LBUSD evaluation systems and support principals in enacting 
effective evaluation procedures and systems at their sites. 
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Professionalism, Disposition, and Ethics 


Model the attributes of an effective LBUSD leader, including demonstrating ethical actions, positive 
demeanor, collaborative working relationships and a connection to community agencies that support 
schools. 


Model a positive approach to interacting with all principals and district staff, including struggling principals. 


Exercise confidentiality and good judgment in all aspects of work. 


Strategy and Planning 


Contribute to a district vision of teaching and learning, based on data and focusing on equitable access, 
opportunities and outcomes for all students by participating in discussion and planning with multiple 
departments. 


Support principals in developing a school specific instructional vision building on existing practices and 
student needs. 


Use data to differentiate support to principals. 


Organization and Management 


Demonstrate, through daily work, that the needs of school staff is the highest priority of work. 
Establish or contribute to systems, structures and processes to ensure efficiency of daily operations. 


Support principals in making decisions about their site discretionary funds. 
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Table A.1. Item-level construction and reliability of Training Quality scale 


Reliability 
(Cronbach’s 
Subscale alpha) 
Alignment with PSI =e “Specifically enhanced my capacity to develop principals’ instructional 0.71 
goals leadership 
e Helped build a learning community with my fellow principal 
supervisors 


e Provided opportunities to share specific practices with other principal 
supervisors in my district 


e Was geared toward implementing district initiatives and programs 
e Helped me understand district procedures 


Job-embedded e Provided opportunities for me to receive feedback on my practice 0.89 
e Addressed real challenges | face in my role 
e Gave me opportunities to plan my work 


e Was based on problems of practice | face in my role as a principal 
supervisor 


e Provided opportunities for self-assessment of my skills 

e Provided me with actionable tools, protocols, and/or learning 
resources that | can use in my work as a supervisor 

e Provided me with tools to set goals for my own development as a 
principal supervisor 


Activelengaging e Was engaging 0.74 
e Was interactive 
e Allowed me to model practices | learned 


Overall assessment e¢ Taught me new knowledge or skills | didn’t have 0.75 
e Stimulated my interest 
e Was part of a sustained, systematic program for my development 
e Facilitated my overall leadership 


Scale overall 0.94 


Source: DeVellis 2012. 


Note: The study team assessed the reliability of the Training Quality scale using item responses from the second 
round of principal supervisor surveys, conducted in spring 2017 (N = 40 due to missing data). We 
measured reliability using Cronbach’s alpha, a commonly accepted statistic for determining scale reliability. 
Coefficients above 0.90 are considered to have excellent internal consistency, those above 0.80 are good, 
and those at 0.70 or above are considered acceptable (DeVellis 2012). 


The items are from the following survey question: “Thinking about the professional development and 
training for principal supervisors you attended during the 2016-2017 school year and the summer before, to 
what extent do you agree or disagree with each of the following statements?” (1 = Strongly Disagree; 2 = 
Disagree; 3 = Neither Agree Nor Disagree; 4 = Agree; 5 = Strongly Agree). 

We created mean scale scores from individual items by averaging first at the supervisor level and then at 
the district level to create the results shown in Figure V.1. We compared these mean scale scores with 
factor scores and found them to be virtually identical. 


PSI = Principal Supervisor Initiative. 
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Table A.2. Item-level construction and reliability of Principal Evaluation System 
Quality scale 


Reliability 
Items (Cronbach’s alpha) 
e The principal evaluation system is too cumbersome.* 0.74 
e There are too many indicators attached to the principal evaluation system to be 
useful.@ 


e |tis unclear how principal evaluation data are used in this district.@ 


e The principal evaluation system provides actionable feedback to improve my 
leadership. 


Note: The study team assessed the reliability of the Principal Evaluation System Quality scale using item 
responses from the second round of principal surveys, conducted in spring 2017 (N = 611). We measured 
reliability using Cronbach’s alpha, a commonly accepted statistic for determining scale reliability. 
Coefficients above 0.90 are considered to have excellent internal consistency, those above 0.80 are good, 
and those at 0.70 or above are considered acceptable (DeVellis 2012). 

The items are from the following survey question: “Please indicate to what extent you agree or disagree 
with the following statements, based on your experiences in the past three months:” (1 = Strongly Disagree; 
2 = Disagree; 3 = Neither Agree Nor Disagree; 4 = Agree; 5 = Strongly Agree). 

We created mean scale scores from individual items by averaging first at the supervisor level and then at 
the district level to create the results shown in Figure VI.6. We compared these mean scale scores with 
factor scores and found them to be virtually identical. 


4ltem reverse-coded for consistency. 
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